This is one of two of three problems ganging up on you,
You will have to figure out which two.
First off, the Ruger has a properly sized chamber, not big & sloppy to facilitate auto loading like the AR-10.
Not only is the AR-10 too big around, but the back of the case just above the extractor groove unsupported (all ARs have unsupported rear chambers, as well as most other types of auto loaders)
The lack of support for the rear of the case lets the case bloat/swell up.
This makes full length/small base resizing dies a must when you bolt rifle won't accept the bloated case...
The shell holder prevents the dies from reaching the base/head of the case,
While the guide taper/radius in the bottom of the die keeps the die from properly taking the bloat out of the case.
If you get a 'Small Base', Full Length sizing die, most times that will solve the 'Bloating' issue... but not always.
The decapping issue is probably either a short decapping pin, or an adjustment issue. Not all die makers have the same length decapping pins, some are shorter than others, and with older/thicker brass (especially military brass) the decapping pin sometimes isn't long enough.
By the time you screw the sizing ball/decapper rod down enough to get the primer out, the rod is hitting the bottom of the cas keeping the brass from properly bumping the shoulder back resulting in a 'Long' case for the chamber you are trying to fit it in.
When you hit this particular problem, either a longer decapping rod, which isn't always available form every die maker,
Or a universal decapping die which will decap about anything, then you size the brass in another step.
The third issue is AR rifles usually have LONG chambers, this is to facilitate the auto loading process.
When you fire a cartridge the shoulder gets pushed excessively forward, and has to be pushed a long way back into place for a more standard chamber, like your Ruger chamber.
This takes a STRONG PRESS to force that brass to conform to the die.
Flex in the handle, slop in the linkage, flex in the ram, stretch of the press frame & die movement all add up together (stacking of problems) to keep you from forming the brass...
What will help,
To bump the shoulder back just a 'Smidge' farther, use automotive feeler gauges between shell holder & brass, just under the brass headstamp.
This will let you force the brass into the die another few thousandths and might get you where you want to be. ($10 a set of blades, any auto parts store).
Calming over the press is a crude way to compensate for the flex/slop in the press, hard as heck on the press and doesn't produce consistant results,
But often times it will give you that extra few thousands you need to CHAMBER the brass...
I suggest a good quality iron or steel FRAME press, cast aluminum frames often just give up with no warning when you seriously over cam them.
You might get away with it for years, but then you get that sickening 'Pop'...
Some alternative ideas,
I hadn't heard of this for 40 years of reloading until recently, I tried it and it works...
Use a .30-06 sizing die to push case bloat out.
The shoulder/neck is far enough up to NOT contact your .308/7.62x51 shoulder neck, and you can grind off the bottom flair/taper until the die walls contact the bloat and push it back in where you want it.
Grind/file in SMALL amounts until die does what you are looking for and no more.
No decapping pin or sizer ball required... you are only pushing the case back into shape just above the extraction groove, nothing else.
----
This is WAY too expensive for small batch loaders... ($1,200 to $1,400)
There is a machine called a 'Case Pro' that rolls the bottom of the brass between two die plates pushing the lower bloat back in GENTLY where it belongs.
This machine also restores extraction groves & straightens extraction lips.
----
What I can't recommend MORE for you is a case gauge (LE Wilson) so you can compair factory rounds that fit with your sized cases, which will give you an idea of what is actually going on...
A case gauge will tell you VERY quickly where the case is out of specification, and where you need to 'Bump' it back into shape...
It also lets you compair fired cases, to let you know the differences between the two chambers you are shooting your ammo through, and how much they need to be reformed before they will interchange again.
A case gauge, a quality caliper, and a little simple math will tell you what you have and where EXACTLY you need to go.