If you perform the thumb drag test on the cylinder, does the bolt (cylinder stop) engage the cylinder notch properly and does the hand slip off the ejector ratchet to allow the hammer to finish cocking in single action mode? I am trying to determine whether the hand actually pushes the cylinder too far or if the cylinder is just coasting over the bolt inertially? In the latter case, check that the spring pushing the bolt up has adequate tension. Taking a diamond hone and putting a chamfer on the top edge of the bolt on the side opposite the direction the cylinder notch moves toward it from, but not deeper than about half the depth to which the bolt engages in the notch, may allow it to start moving in before the cylinder can coast past it.
Taking metal off the hand to reduce single action hand engagement risks disabling double action operation since the hammer doesn't move back as far in double action operation. If you do reduce the hand, do the thumb drag test double action to make sure you don't reduce the metal enough to disable the double action function.