Ignore it. Sling swivels click around, slings slap, levers clack, bolts clunk, the AR rattles. It's a combat weapon and made that way.
Nobody ever heard it and shot the guy carrying one, all the other bashing around to traverse cross country is much louder.
What's funny, I spent 22 years carrying one in the Reserves and never heard it until I built my own. Until then all I heard was the upper and lower clicking a bit, or the sling.
What our instructors and opfor heard was a "herd of buffalo stampeding thru the woods." Every single time. It's like they rehearse and practice saying it, some kind of mantra.
Now my sons hear it when we hunt, because I say it.
Don't let it bother you one bit, the buffer is doing exactly what it should.