From my experience with slugs, you will be very, very lucky to group better than 12 to 15 inches at 100 yards. There are a lot of factors, the two biggest being open sights coupled with the inherent poor accuracy of the "rifled" slugs. I have no doubt that one could hit a deer, but hitting it where you want to hit it is a different matter.
Best bet would be to buy a rifled slug barrel, and use sabot slugs. You will be able to get 2" groups at 100 yards. But this is an expensive proposition - a rifled barrel will run you $200, and sabot slugs are $2 per round.
I got a rifled paradox barrel made by Hastings for $202 from MidSouth Shooters Supply. I believe that they have rifled barrels starting at $150 or so. With a non magnifying red dot scope ($35), I have no problem getting touching holes at point of impact at 50 yards. No signs of the recoil being too much for this bargain basement red dot either.
Next bet - a 20 inch rifled sighted barrel with choke system. IC or modified will help, as will rifled chokes.
If you want to keep your hardware the same, practise stalking to 50 yards and shooting at that distance or less.
Good luck,
Bowser.