Wow, what a mouthful. Also, I don't agree. My experience this Fall was that if I'm racing every week, I can work out once or maybe twice a week in addition to racing. There are four other days in a week! My experience is that I only need to take one or sometimes two days off cycling in a week that has racing and working out. So there are still two or three days of cycling left unaccounted-for. These days can be recovery rides, or just whatever. They're still miles, and they're still a higher training load than sitting on my butt, but nowhere close to what I do to myself if it's a specific workout. I would say that that describes going riding with my friends just as well as it does a recovery ride.
Maybe my friends are just slow. Or maybe freaking out about junk miles is a little silly. I guess that ultimately race results are the proof. My race results improved all last season and I achieved my goal. Things might change as I keep going racing and work my way up the pack, but I hope that I never have to give up junk miles like rides I enjoy, rides that keep me out of cars and busses and rides that prevent me from riding bikes with friends. I think that if I ever do "have" to do that, it would tell me that I should reexamine my motivation and what it is that I want out of cycling. Or maybe I'll keep doing all those, keep doing training rides a few times a week, and beat a few more people every race until someone makes me upgrade and I start from the bottom again.