I think it is because you may be "overtraining". Now I'm usually not one to blame anything on overtraining, I've done some ridiculously high volume arnold like routines and made gains, but it sounds like this could be the cause in your case.
To fix your sex drive there are a few things you could take. All these are OTC.
Arginine Take a few grams a day, at least 3 or so, this helps the blood flow to the penis
Yohimbe Take no more than 40mg of yohimbine, yohimbine is the alkaloid found in yohimbe which will help get blood flow to the penis, try to buy something that says how much yohimbine is actually converted from the yohimbe
Horny Goat Weed(DON'T buy the stuff at walmart) This stuff works well also, just take the recommended dosage
Zinc Take no more than 100mg a day, this also helps increase the blood flow
Post your current routine. Also how many hours of sleep are you getting a night?