Are you casting your own boolits? If so, as has been stated before, either size them to .001 or .002 above your bore diameter, or just don't size them at all, depending on what size they drop from your mold.
In other words, you want the lead boolit to be at least a couple thousandths over bore diameter. SLUG YOUR BORE. It doesn't take long, even if you push three slugs through (as you should). Measure with a micrometer to get the best accuracy.
Also, thoroughly clean your bore, then inspect it with the brightest light and most magnification you can... is the bore rough? Then lap it. Use the slugs you pushed through, and lap it with all three of them. Clean thoroughly, then load up a good 350-400gr boolit with 14-15gr of Unique, no filler, and see what happens.
Hardness of the boolit is not as big an issue at 45-70 velocities as is diameter. I shoot my Lee HB 405gr boolit pretty soft, at about 1300-1400 fps with no leading issues at all. It's also just about one-hole accurate at 50yds, and about 1 1/2" at 100yds. This is a Marlin 1895 SS though, not a Handi-Rifle, but the potential should be the same.