The groups result from a Gaussian bivariate distribution of random numbers with equal standard deviations on both axes. It is the combining of two bell curve distributions (the "normal distribution"; the most common kind in nature) on axes that are at 90° to each other, with the random values along each axis being normalized to the Gaussian distribution probability, but found independently of one another. The location of each hole just uses the resulting normalized random values as cartesian coordinates. Using the same standard deviation for both bell curves makes the group shape round on average rather than oval. This means it assumes all error sources contributing to the size of the groups are also round or else that their contributions to shot deviation combine to result in a round group. So it's sort of like firing from a machine rest in zero wind.
More on that statistical approach is in the Ballistipedia,
here.
You are correct that real shooting often has non-random variables, though they are usually actually biased random variables. For example, the general direction of a flinch is usually predictable, even though the exact magnitudes and exact angles away from POA of individual flinched shots have random variations among themselves, so its effect can be modeled as an additional pair of bell curves along and perpendicular to the general flinch axis with the center being offset in the direction of the flinch and the two curves having different standard deviations so the effect of the flinch is to produce an elongated ellipse combined with the rest of the target. The reason bell curves are still involved is that nerve firing and muscle twitching both have noise that is random.
The other complicating factor is that random error sources don't add directly. Two random error sources can subtract from each other's errors just as easily as they can add together in any given shot. Similarly, the direction of each error can be at right angles to one another and thus neither add nor subtract along each other's axes. So how do you allow for this? Because the angle of error away from the POA is random, going all the way around the clock, the average direction is halfway between and is thus at a right angle, so the contribution act as the cartesian X and Y coordinates, with the combined effect being a distance from the center that you find by the Pythagorian rule, the resulting distance from center being, on average, the square root of the sum of the squares of the two contributing error distances from the mean. Thus, if I have one source of error that makes one-inch holes when it is acting alone and another that also makes one-inch holes when it is acting alone, and I then combine the two, the resulting average group size will be the square root of two inches, or 1.414... inches. The brain is tempted to say it will be two inches, but to get a two-inch group, the two sources of error have to act with maximum amplitude in the same direction for each of the two shots that define the extreme spread. It's actually a highly improbable coincidence. It will happen once in a while, but not on average.
Where the above becomes important is when you look at fine-tuning steps. Suppose, for example, you have a source of error like bad bedding that produces one-inch groups when acting alone and another source of error, like uneven neck wall thickness that produces 0.1" groups when acting alone. So you combine those two errors by the Pythagorian rule, and you get:
Group size = √(1²+0.1²) = √1.01 = 1.005"
That means if you start outside turning your necks to eliminate your 0.1" source of error, your average group will only shrink by 0.005"! You basically won't be able to measure that you've made any difference unless you shoot really huge group samples and are likely to conclude outside neck turning does nothing.
The bottom line is that a lot of fine-tuning people do becomes invisible due to larger error sources covering them up. So you could be doing some good with your extra measures but not see any obvious effect, even though, every once in a while, you'll get an extra point on a target due to that 0.005" being just enough to bring you a scratch of the next higher scoring ring.