I agree that everyone should be as fit and healthy as they can.
I agree that everyone has an optimal level of fitness, weight, and body fat percentage where they are most attractive to the opposite sex. I believe anyone looking to impress women or just boost their confidence would find it advantageous to try to reach that point.
What I don't believe is that changing your body is the answer for every guy. I think some guys can reach that point, but still have difficulty attracting women. I think it depends on a lot of different factors. Maybe it would work for most.
Regarding jobless guys who were good with women, I've definitely known some of these. In fact, the traveling vagabond who travels through town and seduces many wives is a classic literary character. As you note, many of these guys are not that well built. Most may be fair of face.
I honestly don't know what the appeal of these deadbeat Casanovas is, but it is clearly there. I think maybe there is some appeal in the free lifestyle they lead, the fact that they are not beaten down by the daily grind of 9 to 5, and the fact that they have 24 hours a day, seven days a week to practice and ply their hedonistic ways.