I have had this problem in the past with certain women. Usually it's the ones who I find extremely sexually attractive and there is lots of touching and holding and rubbing between us just walking around, and walk around with a bone for hours while we are doing other things, eating, partying, shopping, etc., then by the time we start up the sex, it might be late, tired, drunk even, I'd had a bone for 3-8 hours or more and it easily went flaccid when the sex started up. The fix for me was to have sex soon after seeing them THEN go do whatever, then the sex was easier and more relaxed later. So maybe try having sex sooner after you get together before getting involved in a bunch of other stuff. YMMV. Good luck.
EDIT: I don't advise starting up with the viagra or any other meds at your age unless this problem gets worse. Don't smoke, don't do drugs, eat right and exercise first.