I have a tac pistol grip fore end on my Defender. I did not install the original spacer. There is no play/slop to the rear on my pistol grip with a round chambered. That spacer is only 1/16" thick and I don't see how that would account for the play you have.
My fore end came with end caps for a Win, moss, or Rem...could one have gotten mixed up when installing? (I know, stupid question?)