How many rounds before I trust a defensive gun? It varies...
If I'm testing out a new gun and it jams (especially if it jams more than once), it may take many hundreds of rounds without one hiccup before I'll trust the gun.
If it is a reputable brand, and better, a brand I already have good experiences with, it won't take so much. My S&W M&P40c was carried for the first time after less than 100 rounds. I was tempted to use my CZ P01 before it ever had any range time, but of course I didn't.
Typically, I only shoot FMJ to get a feel for a gun and to have some fun. The real test, and usually before shooting any FMJ, is I usually buy several boxes of cheap practice JHP: Fiocchi JHP, WWB JHP, S&B, etc. I also buy a couple boxes of quality defensive ammo of different types. If I don't shoot all the boxes, I will at least take a sampling out of several different loadings. If it is a reputable gun, I'll usually trust it after 100-200 rounds of various JHP. I won't use any particular ammo that I haven't shot at least one, and preferably more, 20-25 round boxes of ammo.
A gun brand, or style of gun with a mixed reputation may take a little more. I have a Rock Island Armory 1911 Compact coming soon from Buds. RIA is a budget brand, typically well regarded, but a budget brand. 1911s can be a little more problematic than more modern designs. Compact 1911s can be more finicky than other 1911s. So, that combo means it will probably get 200-300 rounds of FMJ plus 200 or so rounds of various JHP before I'll trust it in my holster or for home defense.
Of course, after it has passed the initial test, I continue to shoot various JHP (including my defensive ammo) periodically. Most range trips will see at least one box of "generic" JHP (the cheaper stuff: S&B, Fiocchi, WWB, etc) and usually at least one mag of my defensive ammo. Plus of course FMJ for cheaper practice. If there are any bobbles in a range trip, the gun usually starts over having to prove itself again (depending upon how many reliable rounds it has fired before then, it may only need a box or two of the cheap JHP to prove itself, or it may need several hundred rounds of mixed ammo before I trust it again).