I swam for about 20 years and competitively through college (Division I).
I didn't strength train specifically for swimming in any serious way until college. But let me tell you, once I did, I did see a drop in my times. It's a trade off, however; for added strength, you'll lose some flexibility, which is a key component of a good swim.
The path to becoming a better swimmer also depends on your experience level. If you're a relative newby, you'll make the most gains by perfecting your technique and building your stamina, as mentioned earlier in this thread.
Technique can't really be described adequately in writing; it's something that you need to have taught first hand. Find an experienced swimmer and have them critique your stroke.
Building stamina in the pool is similar to building stamina in any sport. It takes a combination of long distance training, interval training, and sprints.
The mechanics of swimming and the medium that you're in (i.e., water) are unlike anything you experience in any other sport. So, even though you may be in good shape on land, that may not translate well to the pool (and vice versa, actually).
You're also hypoxic (reduced oxygen), which is what leads to the burning arms and legs taht you're experiencing. Working on the two concepts above (perfecting technique and building stamina) will both help with this.