I hunted for 20 years with the .30-06, then the 35 Whelen and the last 10 with the .338-06. So I guess that qualifies as owning all three. The .338-06 pushes a 10% more frontal area slug, at about 300 fps faster than the .30-06 given the same weight. That is nothing to sneeze at. The performance on game, mostly mule deer and elk have resulted in some of the guys I hunt with going to the .338-06.
At 500 yards, the 225 (Nosler with a BC of .550) .338-06 has 47" of drop and 1900 ft-lbs of energy. At 500 yards, the 220 grain (Nosler with a BC of .351) .30-06 has 90" of drop and 912 ft-lbs of energy. Maybe for little whitetail deer that does not make a difference, but that is huge in my book. I've looked at the bullet performance from 40 to 505 yards on over 30 animals, mostly mule deer and elk, with all three calibers and the performance difference is obviously in significant favor of the .338-06 followed by the .35 Whelen, and then the .30-06.
To say the .338-06 recoils considerably more than the .30-06 is where I really disagree. It is a little more than the .30-06, but nowhere near the .300WM. Maybe you had ill fitting stocks. The .35 Whelen is a little stouter than the .338-06, but still not close to the .300 WM.
jmr40, you might want to review your notes, because I think everything you said is pretty easy to prove wrong, except that yes, the .338-06 is a reloading proposition.