Hey
@Browneyes741 I've hesitated to chime in before now but having read many books on interpersonal relationships, human behavior, romance and the like, it's possible your telling him about your sexual fantasy triggered certain emotions in him (i.e. sexually) that he didn't feel comfortable with given he has a girlfriend and well, in general.
It also might have resulted in him feeling somewhat vulnerable and sometimes, for some men (certainly NOT all), feeling vulnerable is an uncomfortable emotion, it can cause anxiety and trigger a "flight or fight" response. In your guy's case, his response was to get angry at you (fight) as you were the one who essentially triggered the emotion.
He may have felt you were being a tease or leading him on, even though that was NOT your intention. So if he was triggered emotionally and sexually by your fantasy comment, his anger at you makes sense.
Below is an article from Psychology Today discussing the correlation between vulnerability and anger you might find interesting and give you a different perspective:
How people in pain can release anger and improve their relationships.
www.psychologytoday.com