Modern ammunition won't cause damage to a barrel other than normal wear.
Lead, copper and carbon fouling may cause accuracy issues if there is sufficient buildup, but they don't cause any damage. Conceivably, in extreme conditions, (perhaps lots of humidity) fouling in the bore could hold moisture and make the bore more prone to corrode, but in practice that's unlikely.
Cleaning a rifle barrel properly won't hurt accuracy, but it's common for the first shot or two from a freshly cleaned bore to shoot to a slightly different point of aim until the bore gets a little bit of fouling buildup.