In Colorado we don't worry so much about rain as we do snow, but snow does melt sooner or later ..... we all wear a Gore-tex fleece (as much for our own protection as that for anything else - but the fleece "silence" does work).
The fleece is mostly silent (especially with rain coming down on it - doesn't make near the noise as does the standard "nylon, hard-shell-type" stuff ... ) & the Gore-tex provides all the weather protection you could want. As an aside, we've bought into the Cabela line of fleece Gore-tex ....
We "bow-hunt" with rifles, doing the "sneak 'n hunt/wait 'n sneak & hunt" method in the dark timber & have shot most elk/deer at under 30 yards.
All rifles are standard/stock wood, free-floated & glass-bedded. Extra care is taken on doing a polyurethane coat
or two (to make certain the wood is sealed against the weather & & a "dusting" with 4X steel wool to break up the shine & then coated with a good canubra wax [any good paste wax works] coat (or two) on all wood & metal surfaces (just for grins) afterwards (don't forget the 4X steel wool afterwards to break up the shine).
Point of aim isn't a real biggy (due to the usual short range shooting) but I've yet to have a wood stock "go south" using the above.
Have yet to have to re-zero any rifle due to inclement weather, but still .... under 30 yards in dark timber? But too, I've yet to have a rifle that wouldn't re-zero at 300 yds using the above methods.
Zip for worries to date ..... (& that GT fleece is certainly very good stuff!) .....
Do a good free-float, wear GT fleece & sneak. Who could want for more?
BTW, I'm a "mouse-pointer-type" too & when the weather's the worst, you will (mostly) have the woods to yourself - all the better.