I recently got my CHL ... What is a good middle ground between being able to draw quickly vs being comfortable enough to carry everyday?
In my experience, maybe 10% of people who start carrying do so more than a year. Many of them don't even make it a month.
The first rule of a gunfight is "bring a gun." If the gun and holster are uncomfortable, they'll wind up getting left a home more and more often, until you go from "always" to "when I think I might need it" to "I wonder where I put it?"
If the gun isn't reasonably comfortable to carry, chances are you won't carry it long. Therefore, I rate comfort over speed of draw. Shooting situations seldom occur instantly; crank your situational awareness up a bit and give yourself time to draw if you need to.
Practice probably counts for more than the difference between the type of holster and location.
There was a surveillance video last year, an off-duty Chicago policeman and his wife in an elevator. They were descending to a parking deck. It was winter, and the guy was moving his gun from his waist inside his overcoat to outside in a coat pocket, which in my opinion was an entirely reasonable thing to do. He fumbled the gun, lost it, grabbed at it (which we all know not to do, but the reflex to save an expensive piece of equipment is hard to override) and managed to shoot himself when he grabbed it. Not fatally, fortunately. The point was, it sure didn't look like he was used to dealing with a gun and an overcoat at the same time. "Don't be that guy." Practice with any clothing you might wear.
Some people can use almost any holster. Some people have to search for one that they can live with. If you have problems carrying, they're probably holster-related. It won't take long before you find out what you *don't* like. It you have a local gun shop they might have used holsters for a decent price. And they show up on eBay and Gunbroker often. What a holster looks like is irrelevant; nobody else should be seeing it.
No holster or carry position is perfect. You have to decide what tradeoffs you can live with. Generally, you find out by spending more money on holsters than you expected...