I'm in the same boat as the OP; I've got three plastic guns (no idea why people insist on using the term 'polymer') and I'm done with that. They work well, yes, but that's all. I never look at my Glock or Shield and think to myself, "wow, that's a nice gun." Ultimately, in my eyes, they lack character. They're like Honda Civics; nice little everyday commuters that will run forever with minimal maintenance, but if you're a car guy a Civic is just blah.
On the other hand, I love the lines of a 1911 (without the rail) or a Colt SA revolver, or a S&W M29; they just speak to my sense of aesthetics (i.e., 1965 hi-po mustang fastback). And I believe this is true of most gun people.
As an example, when I go to the range with my Glock, no one ever comes by and asks me if they can get a closer look at it; it's a Civic. But if I show up with my dad's 50's-era 1911, or my Vaquero, people will practically stand in line to talk to me about them.
Now, it's a given in my mind that most plastic guns will perform as well or better than any all-metal ones, and they're lighter, easier to maintain, etc. But, and you'll excuse me for saying so, but if I'm going to lay down several hundred bucks of my money, I also want the thing to give me some sense of pride-of-ownership. 1911s, Hi-Powers, revolvers, etc. give me that as no plastic gun can.
Of course, that's just my opinion, and as the saying goes, everyone's got one . . .