A cab from LaGuardia into the city will cost you from $60-$80.
The best way to do it if you don't have that kind of money is to take the MTA Airport Express bus. It's $12. Just go out to the loading/pick up area and ask where the MTA Express into Union Station is. It stops at Union Station and the Port Authority. It takes maybe 40 minutes if traffic is light to get into the city from LGA.
Then just get off at Union (since you're going into the lower half of the city, that's where I'd get off) and look at the subway map to find out which line(s) to take. Or just ask the attendant in the booth at the subway entrance how to get there. It's really less daunting than it seems when you figure it out.
You could also just take a cab, I'd estimate that it'd cost you around $20 from Union Station.
Let me know if you have any more questions about the city.