"Frankly, I still feel like I need 3-4 days to recover after a 45 minutes erg workout"
The huge 3-4 day recovery required tells me that you pushed well past a healthy exercise duration, and into fatigue in your 45 minute workout. It also explains your lack of "improvement" in a 2K at the end of a workout.
Use a heart rate monitor for smartest, safest training. I did, and still do, since becoming a bypass recipient 12 years ago.
Watching my heart rate ALWAYS, during exercise post bypass surgery, I discovered a rising heart rate after a long stable period of aerobic work was an indicator of coming fatigue. And like yours, it took a couple days to get past. I used this indicator to back off in that workout, or start an active cooldown. If you get the right brand of heart rate monitor, it will give you Training Effect. The higher the value, the tougher the workout, but also, the longer the recovery time.
Result ? No fatigue. And, and over time, I allowed the rising heart rate to continue for a few minutes before cooling down, sometimes just backing off for a few minutes, then pushed on again. This helped increase endurance, and to raise my max heart rate.
Also, be sure to warm-up gradually before a heavy workout, or 2K piece.
The warm-up: causes partial shutdown of internal organs and shunts blood from them to skeletal muscles, and improves blood flow to the heart itself. I was told this by two doctors, who both know both the heart and exercise. One is a champion Master's rower.
Skip a proper warm-up, and you make sure you will NOT have a true personal best. Plus, working more on adrenaline, than good sense, one raises their blood pressure excessively, which is NOT a healthy thing to do.
If you still intend to do a hard 2K at the end of a workout, at least give yourself 3-4 minutes of easy rowing first. You might try flirting with your anaerobic pain, stay just outside it. I found that if I backed off a touch, the work felt easier, and the split time went down a little, and I did not tire as quickly. 2K's are also about pacing, not just all out effort.