There isn't a holster that can prevent someone from getting the drop on you.
Nevertheless, I can see drawing from an ankle holster not working for some people. For others, with good technique, it can be very fast. The fastest technique is to bring the leg up and the hand down at the same time. A big advantage is the holster is pointed in the direction of the target before the gun is even drawn. I am totally convinced from a timer beep to muzzle on target it is as fast as any other technique where a concealment garment must be swept.
Of course, reality is not always like practice. In some circumstances, a person would drop or take cover or concealment before they attempt to draw. Getting down before you draw is slower to get the muzzle on target, but a person may do it anyway, because drawing isn't the only thing we think about. There are some circumstances where ankles are inaccessible. One person wrote about needing to draw when being pressed in a large crowd. But ankle holsters are easier for some to reach when seated in a car compared to carry options where one is sitting on the gun.
With ankle carry it is so easy to achieve concealment without printing even when you're wearing nothing more than a shirt and pants and you can even tuck the shirt in. Quick-draw isn't the only concern, especially for those who carry to "have and not need rather than need and not have." It is a contingency, and compromises are made, those associated with ankle carry being the least of several others.