Here's my opinion on the subject.
In an earlier era of when police officers were respected citizens who wore their sidearms as a badge of office as much as for actually using them, and back when officer "O'hara" the kindly beat cop who everyone knew, respected and liked, who carried his revolver every day yet in most cases never had to draw it for his entire career, and before psycho, Billy bads, power hungry, egotistical, militarized alpha dogs started getting hired in place of the former officer "O'hara's", the .38 special was (and still is) entirely adequate as a sidearm with sufficient knockdown power. At one time in this country, LEO's were respectful citizens who commanded respect not because of the uniform they wore, but because of the kind of person they were that everyone on their beat knew. Another important point is the function of the military is to kill the enemy, wherever they can find the enemy, and kill as effectively as possible. That was not the function (nor should it be) of LEO's past or present. Their purpose was and is to uphold the law and only to kill if absolutely necessary. So the thinking of that time by dept heads may have been that officers did not need a large bore military sidearm that kills easier than the .38 special.
The .38 special usually was much lighter and smaller than the big bore sidearms. Many depts had/have their own armorer who also reloads for the dept, so the costs of brass and projectiles of the .38 being smaller were cheaper also than larger brass and larger heavier projectiles since there was less metal and powder content to purchase and use.
It is amazing to me, but I have found that many people do not realize that the .38 special is more powerful than a 9mm. The projectile of a .38 and a 9mm is interchangeable. You can pull a .38 out of its case or a 9mm out of its case and use them in either case and they will work. The projectiles are pretty much interchangable but the .38 special case has a larger case capacity than the 9mm, thus it can be loaded more powerful even within safe standards for the revolver.
Back in those less complicated, more sane days of our country (compared to today), before LEO's lifted weights, had an "us against them" mentality and dressed and armed like Seal team six, the .38 special was the perfect sidearm for kindly officer "O'hara". It fired a cartridge that was more powerful than the 9mm, yet was still very controllable and relatively light in weight compared to larger bore revolvers. Although the .38 could be no doubt a killer, if not stuck in a critical body part, a person could be incapacitated and still survive to serve their dept to society. Whereas with a big bore their chances weren't as good. If one follows the real function of law enforcement, (unlike the military) it is not to kill if possible, but to enforce the law. The law is better served if a criminal can be incapacitated without killing them. And I say this as a former Leo myself, somewhere along the line that mentality has been lost on our current militarized law enforcement agencies. Perhaps the movies are at fault with de-sensitizing death and glorifying killing. Perhaps it's just that society as a whole has changed, but it is obvious that law enforcement agencies as well as individual leo's today have a different mentality than they had in the earlier part of the 20th century. Today officer "O'hara" doesn't just shoot you once perhaps wounding you on purpose so you can survive. Today he shoots you multiple times (because he was TRAINED TO) whether you were a threat either before or after his first shot or not. Then in spite of the person being mortally wounded and dying, the officer's backup will shoot you multiple times too. That's how it is today in our trigger happy, militarized police society. Here's an example of an innocent person being shot seven times by an officer and his backup officers.....
http://www.youtube.com/watch?v=A2nUCx6Ln3w
Concerning weight and size. Compare this smaller .38 special Colt to this much larger .45 acp 1917 S&W. Now it might not have been problem back in those days for some officers to carry a .45 acp 1917 S&W all day, day in and day out, and might not EVER have to draw it. But for the AVERAGE officer, the .38 special's lightness, more controllable recoil and smaller size was preferable and completely adequate for the job required. I believe it still is and bemoan the militarization of our police forces as a portent to a future police state. Anyway, compare the sizes below. The Colt feels like a kids small toy cap gun in my hands compared to my 1917 S&W.
I'm a former Leo. If I had been one back in those days I would have carried the .45 acp 1917 S&W because it fits my big hands better and I'm a big guy and used to it and could more easily carry it on me with comfort than a smaller officer could. I hate it when my pinky falls off the smaller revolvers with smaller grips. But not everyone is like me. Also, back several generations ago, people were less big in stature generally. Statistics show that we as a people generally have gotten larger than our fore fathers. I think it's because we are what we eat. And back in those days they didn't have steroids in the chicken and meat. We ingest those steroid fed foods and over a period of growing years it makes us bigger too. A 6 ft tall man was not the norm back in the old days. Even though I'm 6 ft tall, my father was 5'8" and my grandfather was even smaller. Granddad in his growing years from 1898 to 1918 didn't eat any steroid fed chickens or meat and dad didn't either in his growing years of 1923 to 1943 because steroids weren't as widely used in chicken and beef back in those days. But I did eat them in my growing years of 1953 to 1973.
So for all the above listed reasons, that is the reason for my opinion of why the .38 special was the choice for law enforcement for such a long time. I think it still should be.
.