I've seen velocity vary 80 fps just between having powder over the primer or down over the bullet when it is fired. That was with 30-06 1964 National Match ammunition (about 83% case fill). So you have to tip the gun to get the powder in exactly the same place every time or use a slightly compressed load so the powder gets locked in place.
Several other factors affect it. Fouling building up in the bore can change it. The temperature of the gun can change it. Variable crimp and irregular brass internal volume can affect it. But the biggest single factor in reducing velocity spread, after powder position, seems to be the primer. If the primer is not seated optimally you get more variation than if you do not seat it properly. Sometimes significantly more. Some loads like some primers better than others, so it can be worth trying more than one brand and trying both standard and magnum primers to see what works best. Irregular crimping will also contribute variation.
Optimum primer seating will mean reconsolidating it. The primer's primary consolidation is its assembly at the factory. Reconsolidation takes that further during seating. The primer anvil feet need to kiss the bottom of the primer pocket and then the primer needs to be pushed three thousandths further, according to two different sources of information on it, one being Federal (for large primers; they like 0.002" for their small primers) and the other being Naval Ordnance at Indian Head whose recommendation is in the 1982 and later versions of the McDonnell Douglas report MD A0514, which you can find declassified online.
There are tools for measuring this accurately, but for most handloaders, the most important thing is to be aware primers seated that way are seated hard. It isn't easy to squeeze them that much. The
K&M Primer Gage Tool will measure reconsolidation for you directly on a case-by-case, primer-by-primer basis, but it's another cost and it is slow going to use and I only use mine for long range match loads and precision load development. Short of a tool like that, if you seat until you feel the anvil feet just touch down at the bottom of the pocket and then measure the primer height and seat harder until you measure an additional -0.003" deeper primer cup bottom, you will quickly learn the feel of getting there. The depth probe on your caliper or a depth micrometer with a narrow enough rod will make the measurement for you. The caliper is a bit awkward without a
depth measuring attachment that gives it a wider base (there is a $10 one at
CDCO under Measuring/Calipers and Attachments at the bottom of the page).
Still shorter than all that, just try seating the primers hard in the first place and see if your velocity standard deviation doesn't go down (compare SD's and not Extreme Spreads, as the latter occasionally include outliers that are large, and the SD calculation mitigates the influence of outliers on your final number).
"There is some debate about how deeply primers should be seated. I don’t pretend to have all the answers about this, but I have experimented with seating primers to different depths and seeing what happens on the chronograph and target paper, and so far I’ve obtained my best results seating them hard, pushing them in past the point where the anvil can be felt hitting the bottom of the pocket. Doing this, I can almost always get velocity standard deviations of less than 10 feet per second, even with magnum cartridges and long-bodied standards on the ’06 case, and I haven’t been able to accomplish that seating primers to lesser depths."
Dan Hackett
Precision Shooting Reloading Guide, Precision Shooting Inc., Pub. (R.I.P.), Manchester, CT, 1995, p. 271.