It may work, you can always just load one chamber and see how it shoots. I have never tried it myself with round balls but I have used .454 TC Maxi-balls in mine and they shoot just fine, no problem with recoil pulling the bullets of the unfired chambers.
If it shaves a ring of lead as you seat the ball, it'll be ok. If it doesn't shave a ring of lead but you still have to force it in with the rammer, it may still shoot ok.