As long as you're banging broads on the weekdays who cares? I'm self employed and love the fact that I'm not a slave to the weekends.
I remember when I worked for an employer and I lived in an apartment and one morning I was in the bathroom starting the shower and realized that the guy next door was in the shower and the guy below me was in the shower and the guy diagonally down from me was in the shower too. A bunch of middle-class idiots all showering in unison in preparation for the daily rat-race.
Be thankful you don't have to deal with that. Most people are in crowds their entire life. M-F morning commute, M-F lunch rush, M-F evening commute, Fri/Sat nightlife. Be thrilled you're in a position to live life outside that box.