The smaller/lighter the gun in a given caliber, the harder it is to make it reliable. Interesting quote by Les Baer in the January issue of American Handgunner.
In reference to pistols with barrels shorter than 4.25", Baer commented: "Its just too hard to get those small guns to run 100 percent. Some people can come close, but close isn't good enough for me."
I also recall reading an article in Shotgun News some years back by one of the authors (Frank James) who bought a Kahr PM40 (.40S&W version of the PM9) and had trouble getting it to shoot reliably. He had a steel-framed version of the gun and thought he was getting an even better carry gun by picking up the polymer frame version. Unfortunately, even after a trip to a custom gunsmith he finally had to live with the fact that if he didn't hold it very firmly it was going to jam on him. It was bad enough that he could only shoot about a box of ammo through it in a range session before his hands weakened from fatigue and he started getting jams.
At some point, a gun gets small enough that if you try to make it lighter by going to a polymer frame you're just not going to be able to make it work all the time.
My personal lower size weight limit for a polymer 9mm is the CW9/P9. In a heavier caliber I would go bigger/heavier. In a smaller caliber, a little smaller and lighter would be acceptable.