don't take any single anecdotal evidence too seriously
probably brain damage mostly just happens to pros getting hit really hard for a really long time.
but everyone is different, some people just get injured easier than others. and even if you have enough protective head gear on, you could get spine injures from the force your head is taking. I don't know too much in depth about boxing sparring but it seems like you can always get unlucky and get hit hard from a bad angle and injury something, or maybe i watch too much mma?^^
you two may be beginners now but if you like it enough to spar you are gonna improve quickly and start hitting much harder, are you gonna stop there?
i personally am not willing to take that risk at 26 years old, i have too much ahead of me to risk anything about my health, even if its a very small risk.