Like has been mentioned, it's your rifle, your dime, GIT-R-DONE which ever way you feel is best.
Yes it can be done, and it can be exciting to work with a new caliber. Also like has been mentioned you may or may not end up with what you set out to from the beginning.
In my case I LOVE the 25-06. I got mine expressly to reach out and touch things with the higher velocity and a touch heavier bullets than what the .243 I grew up shooting could produce, but with about he same recoil.
That said, my daughter and I fought over that rifle every deer season, and she usually won. Go figure. So I decided to get me one of my own. This time around however I wanted to go custom and build it to preform better with the heavier bullets. I went with the AI version and a 28" 1-9" Broughton 5C barrel. I purchased a donor 30-06 in a new Remington BDL for less than I could have gotten the action alone. Then added the Broughton, a HS Precision stock, Jewell trigger, Callahan firing pin, and a .250" recoil lug for good measure. I had it all put together by a local BR competitor/gunsmith, and ended up with a rifle capable of running 120gr bullets at 3250fps comfortably and consistently grouping around an inch at 300yds, and less if me and the conditions are both in tune.
Yep I spend a chunk of change, but I got just what I wanted out of the deal. I also spent the better part of a couple fo years picking up the pieces a little at a time so it didn't break the bank all at once. To date I haven't even run a lighter bullet through the thing to see how they shoot, or to what insane velocity they might get. The 120's are fast, flat, and accurate beyond my expectations, and the load is two full grains off what I worked up to as a max for this rifle. The best part of it all is they aren't the so called "premium" bullets I am shooting just some Remington CL's I picked up in bulk. The sad part is there aren't any "premium" bullets other than the Partitions in this weight, as most 25's are a 1-10 twist. The bullet manufacturers simply won't build a bullet that they figure won't stabilize in the standard twist available in a particular caliber. When Nosler first came out with their BT in a 100gr I immediately contacted the fellow I dealt with a lot back then and asked about a 115gr version. They said it was impossible to get them to stabilize in a 1-10, and now look at them. I also have some 125 and 130gr custom bullets which shoot just as well and am looking into picking up some others from another manufacturer to try out this spring. I'm hoping they will shoot as well as the CL's and if so, whoa be it on the hogs.
Why I went this way verses the WBY was due to the fact I have a ton of standard 25-06 and 30-06 cases already to work with. I can also use Lapua 30-06 necked down and fire formed as well. I don't have to spend a buck and a half each for cases that I might dump into the tall grass while chambering a second, third, or more rounds and end up loosing it. Speaking of that I can still get a full 5 rounds in my magazine, with one in the chamber. Can't go there with the WBY in a standard magazine.
Mine was setup for reaching out across my back pasture or my friends pastures, where shots might hit 3-700yds on feral hogs or coyotes, and even a nice buck if within a reasonable range with proper conditions. I have no illusions that it can easily do the job if I do my part. Am I satisfied, you betya, and then some.
So pick your poison, just be sure to factor in how much you will be shooting, what the available components will run you to do so, and what your intended goal is when done. Once you have that in mind, go for it.