IIRC, the .300 is about 20% more recoil than an '06. The .300 is not a lot of fun for a lengthy session on a bench rest, but in a hunting situation it's no big deal.
IMO, the .300 would probably give you another 100 to 150 yards of useful range, compared to the '06. You gotta work out your own skill level for field-shooting accuracy and confidence--nobody else can tell you. For elk, I'd say the '06 oughta be a 250-yards or so reliable killer. I've made one-shot kills on whitetails to 350 and 450, but I've been shooting the same rifle for several decades.
The next thing to consider is how much of what type shooting would you be doing? If elk hunting is occasional, but varmint-sized stuff and deer are more frequent, I'd probably go with the '06.
FWIW, Art