I think you have to look for the underlying 'distrust' of non-drinkers, even at an unconscious level.
I hit a great deal of resistance when I suddenly stopped drinking ( or rather more than 2 pints).
You're not one of the lads anymore, too good to drink with us ehh?
With women it can be worse. A common false belief installed in people is 'Never trust someone who doesn't drink' Why?
When people drink they lose inhibitions, in some ways it's a good test of what your lower brain in capable of once the lid comes off.
Maybe you're a scheming, controlling psycho and you know if you drink people will find out, or you're one of those weepy types, how embarrassing.
I just say that I made a decision that I was going to drink in moderation so I could enjoy the night, remember what happened and be able to function the next day.
If that ain't good enough for someone then fine, that's their problem.