MrsSerenity,
The .32 caliber revolver is a delightful gun to fire. Unfortunately, in the .32 S&W Long cartridge it is probably not suitable for self-defense. The .32 H&R Magnum cartridge seems to get a favorable nod from most shooters as being similar to a .380 Auto in power and I'd agree. While the magnums are louder, produce more flash and are slightly more expensive, I think they're worth it for self-defense ammo. I have a S&W chambered for .32 S&W Long and I love it but the cartridge is not up to the task of stopping an aggressor.
If you're contemplating a Taurus model 731SSULC, the recoil with the magnum loads should not be too much different than the .32 S&W Long - my experience, however, is only with the Taurus stainless steel version.
As to the .22 rimfire -- I'd recommend against a Charter Arms (C.A. fans can flame me privately). Most Charter Arms guns I've seen simply don't hold up well to extended use. The advantage of a .22 is that ammo is cheap and it's easy to shoot. To me there's no sense in buying a .22 caliber gun that won't stand up over time. I'd suggest looking at a Taurus 94 instead. If the gun is to be used more for targets & plinking at tin cans, you might be able to find a used Ruger Bearcat single-action to enjoy. More expensive would be the 10-shot S&W Model 617 with a 4" or 6" barrel.
Final thought:
If you purchase a .32 Magnum and fire .32 Longs, be sure to clean each chamber in the cylinder thoroughly with a bronze brush to remove the lead build-up at the front of the chambers. This build-up can prevent you from chambering the slightly longer Magnum round after as few as 5 rounds fired from a chamber.