Someone mentioned something about the accuracy going sour. IIRC, that was because of how black powder needed to be cleaned and the fact that the cleaning rod had to go through the muzzle.
I mentioned it, besides hearing it for years as one of the "hunter myths", I've actually seen it, more than once. And in rifles that were treated no differently than an identical gun in .30-30. It even happened to my father's .32 Special.
Back in the late 60s, my father picked up a nearly new (less than a single box of ammo fired) Win 94 .32 Special from the estate of a cousin. He used that rifle during deer season for the next decade, firing less than 10rnds per year with it, on average. A couple or three to verify the sights were still on, and then a few at deer. Never had a single blackpowder round in it.
And, being that my father was an NRA certified instructor for both rifle and pistol, and taught Hunter Safety courses for decades, and owned and used a number of rifles, pistols and shotguns, I think he probably didn't screw up the muzzle by cleaning it wrong.
Then, one season, he pulled it out of storage, and it shot low. WAAY low. Like 6" low at 50yds, with the rear sight run all the way up. And it did it with the ammo it shot well with the year before, and with new ammo we tried in it. No one, including our local smiths could figure out why. I've also seen other .32 Specials do weird things, usually not a radical shift in POI, but a doubling or tripling of group size, for no determinable reason. Bores look good, crowns look good, the rest of the gun well within typical for a Win or Marlin. Makes no sense, but it does happen. Really rare, totally unpredictable, but often enough to keep an inflated urban legend alive.
And as I've said, I've also seen very worn .30-30s keep enough accuracy for woods deer hunting. All I can think of is that sometimes, when the stars line up something between the bore size, twist rate, bullet shape and speed goes sour, in a particular individual rifle. Sure, damage from bad cleaning, or not caring for a black powder bore can do it. probably the most likely cause of the legend, but I have seen rifles where I know for a fact that was not the case, do it, too. ITs just ..weird. Nothing seems wrong, no way to predict it, never happens in probably 99.9% of .32Spls, but does happen sometimes.
One thing I've learned in over a half century of being around, using, working on, and reloading for many guns is, that while some things are rare, and some things are almost unbelievable, I try never to say "never".