Looks matter. But mostly for online dating. In real life you can find other ways to compensate - game, social value, your presence, frame, etc..
I think it comes down to this: as a man, you have to bring value to the table. This is either: good looks, money, status, game, social value...