On a limited budget, I'd want to stick to one centerfire caliber and one .22lr handgun.
If truly limited, the 9mm is the best all-around semi-auto caliber as it is reasonably effective for SD, it is reasonably accurate for target/range, and is cheaper than about any other auto-loading centerfire caliber.
It is also cheaper than .38sp or .357mag- at least where I live.
If on a budget, lets' keep reloading for another day.
I would want the .22lr to practice and plink, and the centerfire for skill and self-defense.
A handgun and conversion kit has the advantage of the EXACT same manual of arms. However, a decent conversion kit will cost about as much as a reasonable .22lr handgun.
If you have one of each, you and a friend can shoot without taking turns.
As your income increases, you can begin branching out.
You can pick more sizes of same handgun in same caliber [redundancy].
You can pick more calibers of same handgun [control redundancy- but caliber variety].
You can pick different handgun platforms, in either same or different caliber.
My personal experiences lead me to not recommend .40S&W as the ONLY handgun caliber to new shooters.
If you are not familiar with it, it is easy to develop bad flinch/anticipation habits, that can take a longer time to break you of.
Modern 9mm projectile design is something I'd bet my life [or my wife's] on- as it is shot placement that is more critical.
My recommendation:
.22lr first
Then 9mm.
Then .45acp.
Then .357mag
And THEN, decide what you want to wind up with.
Now, if you have the budget, here is another option:
Glock G35/22/23 [yes- it IS .40. Hang on....]
.40-9mm conversion barrel for appropriate length slide [between $240 and $100- depending on brand and new/used].
9mm mags.
.40-.357sig barrel [$225-100]
no new mags.
.22lr conversion kit for top end. [$325-250]
End result? One serial numbered frame, and 4 calibers: for a cost that is less than 2 Glock handguns new.
Or, Glock G22 [I like the sf] in .45acp
.45-10mm conversion barrel.
10mm to .40 conversion barrel [not recommended, but it works].
does anyone do a 10mm to .357sig barrel?
.22lr conversion top end.
The disadvantage of this second option is that there is no 9mm priced ammo involved.
You can do something like this with a .40 BHP, or a Springfield XD, or a Beretta, or a Sig P226/229, or a CZ. You can also do a variation on this with a 1911: .38super, with 9mm barrel, and .22lr conversion top end. Or .45acp with .22lr conversion top end.
And in all of these 'conversion' projects, you could be out $1000-1500.
Or, you could buy a basic G19 and a conversion 22 top end for about $800.
or you could buy a used G19 and a Buckmark/Ruger Mk IV for about $800.
Or,
Or,
Or.
YOU have to decide. Isn't it fun?
Oh, and then you can get into historical collecting too! 9mm handguns from WWII! [walther P38, or Star B are reasonably reliable].