There is a tool made for just this purpose. They make them for Ruger, and Colt style single-actions, and there is one dual end for both. They're not cheap, I think they run 25-45 bucks, with the high price one being the double end one.
You can get them from Brownells. They call it a base pin puller, oddly enough.
I had the same problem with a USFA revolver. The tool did the job without a hitch.