I've owned Glocks, HK, XD, Ruger P90-97 and 345, Sig, M&P as well as several 1911's in 45 ACP. The 1911's with the narrow single stack grip concentrate recoil into a smaller area of the hand had have the most felt recoil despite the weight. The Glock, with it's wider grip, lower bore and more natural grip angle has the least. But any of the plastic guns are very comfortable to shoot and the 1911's aren't uncomfortable.
Some with smaller hands simply can't make the fat grip of the Glock work for them. I've never fooled with a Gen-4 with interchangeable grips. They may help, maybe not enough. I liked the M&P pretty well, but liked the Glock better. Just didn't want one of each is the only reason for letting it go.