And carried correctly, it doesn't.
After over thirty years, and learning to carry with those bigger guns (and a full sized 1911 is one of the lighter weight guns he's carried over the years), my Dad's pretty much got this down. It's not a question of manliness. It's a question of practice and knowing how to do it right.
Which is why the too heavy/too big argument doesn't fly with me. I've seen way too many people who get all excited about their new gun, shoot it and practice with it extensively and then, after a week of carry on a crappy belt, turn around and sell it because "oh it was too heavy". One guy I know is notorious for doing this.
If these people are willing to put in the work on the range and in training, then why aren't they putting in the effort and work into the other half of the equation? Instead, they go out and get some little mouse gun that isn't as effective and they can't shoot nearly as well, which pretty much defeats the whole point, now doesn't it?