I'm less than half your age, so take this with a chunky grain of salt, but I deal with the same kind of issue. This has happened mostly at work, but even in social situations I sometimes give the "don't f with me look" without thinking about it. I'm a little taller than you and weigh more, in my case I think it has something to do with my imposing size. Its relative to the person I'm talking to. A 5 foot nothing chick will be more intimidated by me than a 6 foot 6 inch guy I face playing basketball.
I remember some lady from an elementary school went and told my boss that I gave her an "intimidating look" after receiving some bad news, this was at age 20. I was upset but didn't think I showed it. It was around that time I noticed a trend developing. I work with kids, and they generally like me, but are still cautious not to piss me off. Usually being imposing and intimidating works in my favor, except with my superiors. They think I don't respect them, I just don't like getting jerked around. I've been told many times that in emails I come off as harsh and critical. I don't know why or how that's possible, this comment even comes from people that I haven't met in real life. After time, people find that I'm easy to get along with. I see that I keep people honest though, which I think is a good thing.
I challk it up to people's perceptions more than anything I'm doing. Also, I'd say I'm taller than majority of people that I know at 6 foot something. People assume I'm taller, weigh less and am older than I really am. This has to do with people's perceptions and has a lot to do with their own issues and project those on to me. Somebody that is mad at the world will find a way to accuse me of being unethical. Some people that see themselves as altruistic, see similar traits in me and praise me endlessly.
I'm just approaching my mid twenties, but I've met many people in my line of work. I've been called many things, good and bad. Sometimes people do this just to break you down or get a rise out of you. People tell you that you're intimidating in negative way, so then you soften up to prove them wrong. I hate it when people try that manipulative ****, but Im getting better at spotting it. Its made me simply rely on my own opinion of myself.
In work and most social situations, I've come to the point that I've developed a strong opinion of how things should be done. I base my opinion on previous results that have proven to be effective. I've been in meetings with men twice my age and its a hard pill for them to swallow when I prove them wrong. Its a tough balance between fighting for what I know will work, versus hurting their fragile egos. I honestly don't think I'm better than these guys, I've just been fortunate to learn what I know. I can say the same for what I learned at Sosuave. I'm no better than the typical AFC, I'm just a little more informed. And since I've experienced some things with women, my opinion is going to be more developed than most guys my age.
Anyway my point in all this is that people will say what they want. Usually its nothing more than their own issues and insecurities being projected onto you. All the more reason to be secure in yourself so that no one can effect your self image.