DRT300,
We have been shooting slug guns for 20+ yrs and have seen many different versions of performance. As Z13 mentioned, ensure your scope is fully secured. These things create a lot of force and can mess up a scope. Also make sure the mag tube cap is tightened all the way.
The Buckhammer is a big hunk of lead w/ a large frontal area that will be affected by wind more than a sabot style slug. Remington claims 3 inch at 100yrds, that is likely done with a 24" table mounted test barrel w/ no human contact. Try the Rem copper solids or the Fed barnes expanders, I have had good groups with both of those. Currently I am using the Win supreme partition gold, but I have heard rumors that they may be replaced.
Look at the Tar-Hunt slug gun web site. They used to have some really good info on sighting in slug guns. They focused mostly on lightfields, but the info applies across the board.
Lastly, when doing your range time don't just rest the fore end on the bags or rest. Hold it with your hand as you would in the field, and then place your hand on the support. Big slow projectiles will move the position of the gun before they leave the barrel and that can effect shot placement.
Good luck, shoot lots!