curious
Curious about the age of your rifle and what the round count might be, can you say? Its a bit rare to find a sporter that is "shot out" but I suppose it's possible. The round count would have to be in the thousands, and the rate of fire would have to be high as well.
More likely, I'd suspect the throat length. If you have access to a bullet comparator, get some readings as to how far forward the lands are from your factory ammo. Most factory slugs are seated very deeply in the '06 in my observation, and many sporter throats are very long, usually not a good combination. If the magazine will accept them, and the action feed same, rifles can sometimes benefit by longer cartridge overall length.
If you don't have access to a real comparator, lightly pinch a pulled bullet in a spent case, very long, and chamber it gently. The rifling should seat the bullet. Withdraw this "comparator" very carefully, and note the difference between your experiment and the C.O.A.L of factory ammo. Differences exceeding, oh say .050 may well be your problem. It could be less than that too, say .030.
Another thing you can easliy experiment with is the action bedding screw torque settings. I'd start somewhere around 35- 40lbs on both screws,and run it up to 60 in about 10 lb increments and see how it shoots. If you get favorable results say at 50, I'd try 5 lb on either side to see if it responds further. Too, you could crank them up differentially so to speak, 50 on the front one, say 40 on the rear, that sort of thing, and see if you get results.
All that my result in a lot of shooting. which is expensive these days.The Savage 110 gives you an easy third option. It's a fairly simple process to screw a new barrel onto the 110 action, and pre threaded 110 barrels are available from several sources. That'd be cheaper than a new rifle.
I've got a 06'/110 barrel, very rough, I might even part with, very cheap.