I'm another 'smooth is fast' adherent ...
Smooth, done consistently, can also help make for less 'fumble prone' performance when things are happening away from a controlled safe training environment. Fumbling, and reacting to fumbling, can probably create the potential for unintended and unanticipated events (think unintended discharge, loss of control regarding accuracy, etc).
Proper repetition, done consistently smoothly, can help lead to faster performance of the learned technique.
What's "FAST ENOUGH"?
Dunno. Couldn't begin to tell you.
One of the training classes I attended had some retired cops as instructors who measured the students against their idea of a reasonable relative 'yardstick' for speed.
It was a 'tactical undercover/plainclothes' type training class, meaning the weapons were expected to be concealed under normal working-type garments and drawn from under such garments, when actual working holsters were being used. No contest rigs.
The first day the students were checked for overall individual abilities by being timed during the performance of a drill which required that they draw from concealment (no hedging by prepositioning hands, clothing, etc), move offline at least one step, acquire a 2-handed shooting grip and fire 2 accurate rounds within a COM scoring zone on a 'standard' silhouette target.
More than a few cops grumbled that they were faster using just one hand and firing just one round. I don't doubt it. Drawing from actual real-world concealment while moving offline and acquiring a 2-handed hold, and demonstrating enough control with their shooting technique to fire 2 accurate shots, is probably a bit slower than using just 1 hand and firing 1 round from the hip. Probably a bit more difficult, all things considered.
It was an assessment, not a drill, though. Everyone was timed as a reference point.
At the end of the week the drill was once again given and times checked.
Nobody 'failed', but we were told that the instructors felt that a time of 1.5 seconds probably indicated an overall combination of working skills that would likely serve students well in the course of their duties, presuming they maintained those skills. This was just a test of various skills and abilities, conducted on a controlled range when the students were EXPECTING to draw, present & fire against a KNOWN & IDENTIFIED threat target, after all. Much of the Observe/Orient/Decide process had already been done, so to speak. (Yes, they still had to 'observe & orient' on the starting signal, but you get my meaning.)
There was some noticeable improvement throughout the class, with many now attaining this 1.5 second goal, and a few exceeding it. FWIW, although I easily exceeded the goal on the first day, I did consistently better (faster) at the end of the week. In other words, I learned some things which helped me and which I took back for continued development.
Some still demonstrated times in the 2-4 second range, though. Hey, it was only a 4-day class and there was an obvious (and expected) range of skill levels demonstrated among the attending cops, after all.
After I returned from that class and continued working with under the watchful eyes of the senior instructor I continued to emphasize proper technique and smooth execution without paying strict attention to the timer. I felt increased speed would come with increased smoothness.
There's no way to know what's ever going to be 'fast enough', especially when considering that reacting and drawing against a threat which has already presented itself puts most folks behind the curve from the start. Action vs. reaction can be problematic and require more than sheer technical skill solutions to resolve ... but a smooth, well-practiced & skilled technique is probably always going to be preferable to a fast, jerky, fumbling, jittery reaction done in response to an unexpected (and perhaps 'unbelievable') stimulus occurring in the real world. Just my thoughts, anyway, and I'm nobody's expert.
However, I can think back to instances where I didn't remember even thinking about drawing my weapon against a sudden threat, let alone think about it as it happened.
There's something to be said for repetitive SAFE training done properly and smoothly to develop technical skills ... which are still considered perishable, you know.