Go with the .40.
It would be better for self defense. The FBI got rid of the 9mm and went bigger after the famous Miami shooting, where it took a fifth 9mm hit on a bad guy to shut him down. Trouble is, before the 5th hit, he killed some agents.
Besides, as what happens to some of us, we buy a gun & start shooting it. We like it so much, we may find out about a local range hosting combat style matches, and we decide to give it a try. It's an informal & cheap way to learn how to use your gun safely and accurately (over time) in a defensive scenario. If you go this way, the 9mm is penalized in combat matches, due to the smaller caliber, but the .40 is not. It's manageable. The big thing about recoil is that at first you notice it - but as you shoot the gun more and more you don't notice it as much. I personally have experienced this as I have shot a .445 Super Mag revolver so much in Metallic Silhouette competition, that I no longer notice the recoil - seriously. However, the above seems not to be true about your senses that detect noise and feel concussion - these seem to stay about the same.
Wayne