If you are going to get only one, .45Colt.
.44magnum and .454 and so on all have their place. Given the current reloading components environment that looks to be with us for a while, 45 Colt is the one that can burn almost any powder, and be easily loaded from CAS gamesmanship to powerful enough as a legitimate hunting sidearm anywhere in the lower 48.
No disrespect to the CAS shooters. I love that relaoding components for 45 Colt are so (relatively) plentiful, and I have you guys to thank for that.