Something that hasn't been said, yet
I think kraigwy has probably hit on the biggest part of the problem.
Threegun may be right, too, but maybe not. It's hard to know without watching you shoot, and Kraig's advice would help to fix flinching.
However, something else that COULD be contributing to your problem is trying to grip too hard with your primary hand. When you grip hard with your primary hand, your trigger finger cannot control as finely as you would like.
The reason this would not happen during more rapid, or even rhythmic fire (say one round per second) is that our muscles cannot stay contracted during impact. A friend of mine proved this with an experiment to measure grip strength during a golf swing. At the moment of impact (the time you'd think that your grip would tighten) there is a sharp decline in grip. Your muscles simply give way to even the small impact of the club head on the golf ball. It doesn't matter how hard you try to maintain grip, your body will react by letting go.
During slow fire you have time to reset, and if your habit is to grip too hard you may be doing that before each shot. During rhythmic fire, if you're focusing on other fundamentals, like front sight and trigger squeeze, then your body may not have time to over-grip before each shot.
It may not be part of your problem, but it just might be. As you're working on your follow through, and your trigger squeeze, don't forget the biomechanics that support good follow through and trigger control. Squeeze mostly with your support hand and let the primary hand focus on trigger control.
I hope this helps.