Based purely on the OP, I'd vote .357, due to recoil over long plinking sessions and the potential savings, especially if the gun feeds .38s okay. It's not optimal for hunting, but that wasn't the primary use as originally listed.
...or you could...
Get both!
...or you could...
Get both!