Genetics plays a big part in muscle building, if you don't have the genetics its hard for a person to build muscle. Myself I'm not ripped but I'm not excessively fat either. I'm a 5'9" guy I weigh 195 pounds I maybe have at most 5 pounds of fat to lose and I would be at a fairly low BF%. I used to be 240 pounds had a lot of stress in my life for a good 5 years I stress ate, I didn't have active job either I traded in my construction career for an office job for 12 years.
I'm back in construction so I'm active again, I have to watch what I eat because I can gain fat easily but I also gain muscle mass quickly as well I don't want to get any muscle wise its a ***** to get jeans to fit when your a 34 inch waist but the legs and thighs are tight. I have to wear large shirts my shoulder span is too wide for anything smaller.
To be honest I find women prefer average built guys even when I was overweight I still had women flirting with me, its all about your demeanor if you don't show any confidence your not going to get any attention from women. The way I dress is cowboy style, jeans and button up western style shirt. I'm not some guys that are wearing fvking sweat pants, women like men that dress like men. I find women can be intimidated of me because the way I'm, most young guys I see out there are jam tarts.
Be yourself its all you can do, if your a hard gainer you have to work at it to build muscle, I've never had that problem, keeping myself from getting fat is my problem. To gain muscle you have to eat, to keep yourself from getting fat you have to work out or be active.