I think it depends on how you want to hunt. If you really want to hunt with a handgun I'd look hard at one of the single shot pistols such as the Thompson Center and I'd seriously consider it in a rifle caliber. You limit your range a bit, but can do almost anything a rifleman can do, except at somewhat shorter ranges.
If you are willing to accept even shorter ranges the revolvers are a valid option. The 44 mag is a sensible choice, but for deer only a 357 mag is very capable at any range I'd be shooting a handgun at a deer. If you think there is ever a possibility of anything larger than deer, then I'd start at 41mag or 44 mag.