You know, it's really almost comical in how the more things change, the more they stay the same.
For decades, the most popular LE cartridge in the United States was .38 Special. Many were not particularly happy with the performance of the cartridge, but anything with substantially better terminal performance usually required a larger and heavier package than what the average cop wanted to have to carry, more recoil than what the average cop could handle, or a combination of both. The next best thing available, from 1935 on, was the .357 Magnum as it did offer much better perfomance than the .38 Special loads available at the time and, while it did recoil more, could be mastered by most people with sufficient practice. The problem with the .357 Magnum was that the gun it was chambered in were large, heavy, and expensive.
Now, fast forward a couple of decades to the 1950's where two very significant things happened. First, S&W brought out the Highway Patrolman (Pre-M28) revolver which offered everything practical that their .357 Magnum (Pre-M27) did but at a substantially lower price. Also, both Colt and S&W brought out medium-frame .357 Magnum revolvers: Colt with their Three Five Seven (later to become the Trooper) and S&W with their Combat Magnum (later to become the M19). With these smaller, lighter Magnums, a policeman could finally have better performance than a .38 Special without having to pack around a bigger, heavier, more cumbersome revolver.
Over the next 20-30 years, the .357 Magnum revolver, particularly medium-frame guns like the Colt Trooper, S&W M19 and 66, and Ruger Security Six, gradually gained in popularity until the 1970's and 1980's at which point it rivalled, and possibly slightly exceeded, the popularity of the .38 Special. While certainly bigger and more powerful cartridges like the .41 and .44 Magnum had been developed, most police departments didn't view them as having significant enough advantages to be worth the extra cost, recoil, and size associated with replacing their .357 Magnums.
Now fast forward to the 1980's and early 1990's. Most police were quite pleased with the .357 Magnum cartridge particularly since JHP ammunition had become available and for good reason: the cartridge offered very good terminal performance in both the 125grn and 158grn loadings that were popular, the recoil, while brisk, was not so severe that the average officer could not master it out of a service-sized revolver, and the cartridge did not require an overly large or cumbersome gun. What the police did start to become dissatisfied with was not the .357 Magnum cartridge, but the revolver as a platform. There became a need, be it real or percieved, for a sidearm that offered greater capacity and faster reloads than what a revolver could offer. The 9mm was a well-established cartridge having been around since 1902 and had been chambered in guns that were of reasonable size and with capacities 2-3 times that of a typical revolver since the 1930's. It seemed like a natural choice and many departments, including the FBI, began switching to new, high-capacity "wonder nines" stoked full of then-new JHP ammunition.
Jump a couple of years to 1986 and we have the infamous Platt/Matix vs. FBI shootout in Miami. While the FBI made several fatal errors that lead to the disasterous and publicly embarassing shootout, they chose for whatever reason to focus on their ammunition which was 9mm 115grn Winchester Silvertips. They really could not fault the quality of the ammo as it penetrated and expanded exactly as it was designed to, but instead they decided that a loading with more penetration was called for. The first attempt at a solution was to use a heavier 147grn bullet in the 9mm. While this sounded good in theory, the relatively low velocity of such loadings combined with the rather primative JHP designs of the time lead to 147grn loadings that would not reliably expand and gave resultingly poor terminal performance (shades of the old 200grn .38 Special loadings of decades earlier).
While the issues with the 9mm would eventually be solved through better bullet technology and the introduction of the medium-weight 124-127grn loadings, such solutions were still 5-10 years away and the FBI and other LE agencies wanted something better right away. The next solution was to simply use guns in larger already-existing calibers like .45 ACP and 10mm Auto. While these cartridges did offer better terminal performance than the 9mm loadings of the time, they required a larger and more cumbersome gun than a 9mm, had severely reduced magazine capacities, or both. While .45's and 10mm's were quite popular in the late 80's and early 90's, most agencies were never quite satisfied with them.
Now we get to the introduction of the .40 S&W as it was pretty much exactly what cops were looking for. The larger diameter bullet, and correspondingly larger hollowpoint cavity, meant that the bullet could reliably expand at the velocity of the 147grn 9mm and the heavier weight meant that it could penetrate deeper than the 115grn 9mm. The smaller physical dimensions than the .45 ACP and 10mm meant that the cartridge could be shoehorned into a 9mm-sized gun and still retain the 10+ round capacity that LE agencies felt was necessary.The recoil, while brisker than that of a comparable 9mm or .45 ACP, was still mild enought that most officers could master it with sufficient practice and was still substantially less than that of a full-power 10mm Auto. Particularly with the introduction of the 155-165grn loadings, which have ballistics remarkably similar to those of the older 145-158grn .357 Magnums, it seems as though cops have finally found a semi-auto cartridge that gives the terminal performance they feel is needed in a package they feel is reasonable, essentially a semi-automatic version of the older .357 Magnum loadings.
Like the .357 Magnums before, the .40 has faced competition from newer cartridges such as .357 Sig and .45 GAP but also like the older .357 Magnums these cartridges have not offered significant enough advantage to supplant the .40 from its position of popularity. The .357 Sig, while offering some advantage in barrier penetration, is more limited in its selection of loadings, reported to cause more wear and tear on the guns it's chambered for, and just not all that dissimilar in performance to the lighter 135grn .40 S&W offerings. The .45 GAP, while offering nearly identical performance to the .45 ACP, also suffers from limited load selection and has a lower capacity than a comparable gun in .40 S&W.
All this being said, the advantages that I see to the .40 S&W are typically more applicable to LEO's than to a private citizen such as myself. With the advent of better 9mm loadings, the main advantage that I see to a .40 is superior penetration through auto glass which is mainly the concern of a cop. Likewise, I see the capacity advantage over .45 ACP as a more important factor to a LEO who is forced to use a handgun in an offensive capacity. Finally, the size and weight issues with the .45 ACP and 10mm are less important to me than they are to a cop because I'm not forced to carry the Batman utility belt in addition to my gun like a cop is. While a .40 certainly is not a poor choice for a private individual, I personally don't feel the need for one especially when I already own handguns in 9mm, .45 ACP, and 10mm Auto.