I would sarcastically point out the obvious and belittle him. Saying things like "aww that's cute, talking bad about me. I remember when that was cool in 4th grade. Go ahead, i'll wait till you're done." And i would get the girls to chime in and be like "remember 4th grade and how that was cute?".
Most important thing is your frame. That's above all else. You lose your cool and you're useless. Stay cool, calm, and collected through out the whole thing. Make sure you're the one to end the interaction by walking way. And don't worry girls notice when guys are gossiping and sabotaging like females and it's not attractive. So they are making themselves look like jack asses anyways.