Not sure why no one has chimed in yet, but I'll offer the advice that is often given for trigger pull issues:
dry fire -- a lot (take whatever precautions are needed for your particular gun, i.e., snap caps if dry firing without them could harm the firing pin, etc.) -- and of course
make sure the gun is unloaded and pointed in a safe direction!
A couple of drills that were suggested to me: Tape a small piece of paper having a cross-hair drawn on it to the wall a little below eye level. Take your usual shooting stance with the end of the barrel about 1 inch from the paper. Align your sights with the intersection of the cross-hair. Practice a straight-back trigger pull without disturbing the alignment of the sights with the cross-hair. Try it, it's a lot harder than it sounds. Do that until you can keep the sights aligned several times in a row.
If you have access to a laser that you can attach to the gun (it doesn't have to be perfectly aligned with the barrel), practice dry firing while keeping the laser dot motionless. If you're pulling the gun to the left at the trigger break, the dot will jump to the left.
Just my initial thoughts. I'm sure others can give more/better advice.
Good luck!
DogoDon
P.S. I have a similar issue with my Glock 23, and I could
swear that I'm not disturbing the sight alignment -- but I obviously must be, so I'll be working on the same drills.