This is a no-brainer, IMO.
If you have a carry permit, you want to carry as much as legally allowed. If it were me, my biggest worry would be finding a way to affix it to the dirtbike or my person where it won't come out, fall off, or otherwise separate from me. Carrying on my person is fine on a motorcycle... but NOT a dirt bike or 4-wheeler when tearing up trails. Trail riding means that you might get tossed. You aren't riding hard if there's no chance for you to get tossed, it's part of the game. You don't want to land on it, and you don't want it to fall off your person and you don't want it to have a chance to somehow discharge or get damaged.
I'd find a secure way to attach it to the bike.
I wouldn't be worried about any kind of an animal while on a dirt bike. The noise would keep most of them away. The speed and power will get you away from the threat. Unless you are off taking a leak, I just don't see animals being a problem. Drunks or crazy hill-jacks, perhaps, but even then, your best bet is to flee. Any decent dirt bike makes you *GONE* in seconds. Not much on earth that can catch one through the woods.
My final bit: The biggest reason I'd want the carry gun with me while out trail riding is because I want it with me when I'm away from my home, and there's few things in this world I hate as much as leaving a handgun in my car. A handgun in the car is a potential stolen handgun. There's a very few people who have a serious safe that's through-bolt mounted to the frame, but most folks simply have the little metal box with the cable. It'll stop the smash & grab teenage punk who steals CDs, stereos and change from the ashtray, but anyone with a couple of tools is going to take that box. Hell, if they SEE that box, they are going to REALLY want that box. And if your car or truck is sitting next to the woods out in the sticks, they really have the time and space to make sure that little metal box leaves with them.