Just my guess. Neither caliber, or 9mm for that matter, is going to stop a drunk, cranked up bad guy, without a CNS hit. Something like 95% of all crimes are done under the influence, that is for the ones that get caught.
So, we are left with shot placement. I do think the .40 S&W suffers from poorly loaded ammunition, with powder not tailored to it's specific use. It seems people complain of excessive recoil, since it recoils about 100% more then 9mm.
Case is near twice as big as 9mm.
Anyway, accuracy is easier with a low recoiling gun, and, most police aren't the greatest shooters, so, a low recoiling gun, like a .38, is more likely to be placed percisely then a .40, if they find it recoils too much.
Elmer Keith and Skeeter Skelton went through the same sort of progression with revolvers that we are currently with semi-autos.
They started with hot .38's, and the .357 was born. That wasn't enough in some situations, so they came up with the .41 Magnum. As is the problem with the .40 and 10MM, the .41 recoiled too much for your average cop, and,
the guns were big and heavy. Result was the .41 'police loads' were so low power, there wasn't much point to having the .41 over the .357. Same with the .40 S&W, compared to the 9mm.
For people that do shoot, the 10mm and .41 Magnum, loaded properly, give a huge advantage over the .38.
One of the best resources on the web is this site:
http://www.ballisticsbytheinch.com/38special.html
by one of the members here. It gives you a real good idea at what barrel length different factory offerings give decent ballistics, vs. other calibers. You might find that some of the .38 special loadings are pretty potent.
Also keep in mind that the .38 is a pretty big case, compared to the .40, and, it's possible to come up with plus P loads that are right on the heels of some of the .357 Magnum loads.
I've chronographed three loads for my .357 1.8"
barrel.
Corbon 125 grain at 1200 fps, 357
Fioochi 148 grain @ 1131 fps, 357
Buffalobore .38 Plus P 158 grain, 1040 fps.