Aw, heck, I though this post was about smokeless tobacco. (Dipping).
Okay, back on track. Pushing the handgun is a flinch. Like any other flinch, it can be 'unlearned" with a little time and practice. Slow down and concentrate on sight alignment through the trigger break. The sights should remain on target all the way through the shot. It takes concentration, but with practice becomes second nature.
The dime trick that Oldandslow talks about is a good training aid. Another training aid is to use a buddy to help you with your drill. If you've got a buddy filming you, he should be willing to help you with this drill.
With a semi-auto:
Stand facing the target, pistol holstered. Have your buddy standing behind you. Your buddy will unholster the firearm and load it, then return it to the holster. You draw and fire. Sometimes he loads it, sometimes he doesn't. Sometimes your hammer falls on an empty chamber. You won't know until the hammer falls.
With a revolver:
Stand facing the target, revolver holstered. Have buddy manipulate the handgun, placing between one and six cartridges in the cylinder, then return the revolver to the holster. You draw and fire, again concentrating on sight alignment, trigger manipulation, and follow-through. The sights should remain aligned on the target through the hammer-fall. You'll know immediately when it clicks on an empty chamber and the flinch will be readily apparent.
It's not just with handguns, but with any firearm. Flinching is a common malady with handguns, rifles, and shotguns. It happens. I've seen AA Trap shooters develop a flinch and work hard to correct it. Let the shot happen, concentrate on the basics and let the gun do what it is supposed to do. Sight alignment, trigger squeeze, follow through. Don't anticipate the shot, just let it happen.