Ever notice how sometimes real life doesn't work out exactly like you planned?
Or how your gunfights don't go down the way you rehearsed them at the range?
Maybe you guys are less clumsy than I am, but as a guy who has gone down on a motorcycle, been knocked down a flight of stairs, and fallen off an elevated surface or two, I don't assume that if I ever need a firearm I'll be standing in my Weaver or Isoceles stance with my arms raised just so.
For me the chief benefit of a holster is retension - the fact that it holds the pistol/revolver in one exact place and keeps it there whether I'm stationary or falling down, etc.
Every time I've landed hard, I've lost things that were in my hands, pockets (including a folding knife clipped in my pocket during the motorcycle crash that I never did find afterwards) or carried about my person, and I"m pretty sure that if I had a pistol shoved in my waistband and went down during some altercation, that would be skittering off across the parking lot too.
I used to train cops for a living, and even though the gunfights I've experienced were training gunfights, its amazing how many times weird things happened and guys tripped or fell while moving to or from cover, across obstacles, etc. You fall down often enough and you learn that things don't always work out like you planned. Sometimes when you go down your firearm slides somewhere and you either can't find it or can't get there and you learn that having a 2nd firearm is not a bad idea.
It's probably just me, but I carry handguns in holsters that keep them where I want them, regardless of what else happens to me prior to the draw. I've spent too much time picking up crap that went all over after I hit the deck.
Your mileage may vary.