Listening to really loud music does damage your ears. Back in high school the teacher played certain frequencies on some "frequency playing thing" and it got to the point where some frequencies were heard by only 2 or 3 students. The older we get, the more frequencies we lose in our hearing. When we are just born, that is when our hearing is the strongest, loud music and loud noises kill the things that pick them up.
I forget what the physical part in your ear is called. But the point is, loud music hurt your ears, period. Most people won't stop listening to loud music, because it isn't going to kill you. And working out does NOT make a difference, unless you are wearing some sort of headphones that lock onto your ears and they get stuck to your squat bar somehow and you drop the bar to the floor, pulling your ears from your head and leaving them on the floor.
Other than that, loud music doesn't do any worse when you're working out than when you aren't. But very loud music IS bad.