The roller should be trapped between the locking piece (that is pinned in place) and the barrel's recesses. The locking piece should not slide so far forward as to allow the rollers to fall out toward the middle.
If the locking rollers are sliding out (from inside to out) of the barrel recesses, you might be in need of a new barrel or some interesting gunsmithing.