This subject's been up for discussion since barrels were made for projectiles instead of something else. And lots of myths have sprung up through the reasoning of some folks claiming to be knowledgable. Here's my take on it and some points I'll make may dispell some of those myths.
First off, factory barrels in over-the-counter rifles as well as standard government service rifle barrels seem to do it the most. I've had some and every one had the same problem; first shot was off and subsequent ones less so for a few to several shots were fired. Close examiniation of their bores revealed that a squeaky clean barrel with no bullet jacket copper wash was in it was the worst culprit. Some bullet jacket material was wiped off the first bullet fired from such a barrel and it was unbalanced enough to shoot wild as it left the muzzle. Subsequent bullets had copper washed off but less so as the rough bore surfaces began filling up with copper. After a few to several shots, no more jacket material was wiped off bullets and they shot to point of aim as determined by earlier firing.
There's claims (in this thread, even) that all barrels are not the same in their metalurgy (density, grain structure, temper, hardness, and other factors related to uniformity in a round bar of steel). As they heat up from shots fired, they start bending and after a few to several shots, their muzzle points to a different place relative to where the sights point. It is true that some barrels change point of impact as they heat up, but in every one of the several I've had stopped that when one simple thing was changed. The receiver face was squared up with the barrel tenon threads in its front end. When the original barrel was refitted and shimmed enoug to make headspace the same as it clocked in to the same place. That barrel no longer changed impact as it heated up. The reason was there was a high point on the receiver face that beared harder against the barrel as it expanded from heat and that stress point made the barrel bend a bit away from that point.
One post in this thread mentions "When a Gunsmith puts together that perfect combination of parts, fit and finish to make something that's harmonically balanced and even and straight and hits what you aim at first time every time... KEEP IT" What the heck does "harmonically balanced" mean? To me, that's a new term in the language of gunsmithing. I know that every barreled action out there whips at its same resonant frequency for every shot fired because it's the same dimension and has the same material in it and is held in the stock the same way for every shot. It doesn't change as the barrel heats up; barrel steel's that way. Harmonics are nothing more than multiples of that resonant frequency but the amount the barreled action wiggles and whips at those higher frequencies is much, much less than at the fundamental or resonant one. The only difference is how much they wiggle and that's determined by how powerful the load is that's fired in it. For a given load, it's very repeatable from shot to shot. Stiffer ones whip at higher frequencies; flimsier ones at lower frequencies, just like a guitar string. Even a guitar string sounds the same note regardless of how hard it's plucked.
Then there's the situation wherein the person shooting the rifle doesn't quite do it well for the first few shots. I've done that. Took me several shots with a good match rifle to shoot consistantly to point of aim. I used to think it was the barrel needing to be fouled enough with bullet jacket copper. When I finally learned how to shoot correctly starting with the first shot, that same barrel shot to point of aim starting with the first shot from a cold barrel went to the same place as one 25 or so rounds later when it was skin-burning hot. Three cheers for a good barrel that didn't scrape off jacket material any significant amount and what it did was uniform all the way around. Didn't take much cleaning to get that minute amount of copper out of the barrel compared to rougher ones such as regular arsenal or factory barrels.
A good barrel with a bore smooth enough not to scrape off significant amounts of bullet jacket and is properly stress relieved and fit to a receiver will shoot to point of aim from cold to very hot. The exception is a .22 rimfire match grade barrel; they need to be "seasoned" with a few shots after they're cleaned so the bullets leave at the same velocity. Folks have been shooting such barrels in long range matches for decades and they don't change point of impact through a couple dozen shots fired 30 to 40 seconds apart. Some may shoot the first or second shot out a few fps slower than those following as the barrel's not powder fouled enough to create higher pressure. It isn't normally seen at the target at ranges less than 300 yards and's typically less than 1/2 MOA difference at longer ranges.
And finally, the barrel that just changes point of impact as it starts out and goes through several shots before stabilizing regardless of being properly fitted to a receiver and the shooter does all the right stuff. That's a barrel that's not been stress relieved properly. More common with cheap aftermarket barrels and can happen with factory barrels. There's no fix for these unless the barrel maker can reheat treat it and stress relieve it.