Does he pay you for you gas/time??? If not, you're getting screwed.
If this is a once a week thing, then it's no biggy. But if you're hauling his butt to work every night, then you have a problem. Is there public transportation where you live? He can take a bus.
It's the law of supply & demand: If you supply a ride, you should demand some money to pay for it.