Well...that all depends on you really. What do you shoot, how often do you shoot, and how do you like your rifles? If you like em pretty, you can't been a fine piece of wood. No fiberglass stock is going to come close.
As far as expanding/contracting yada yada....*shrug* benchrest shooters use wood (not exclusively). They don't just shoot one part of the year in one place...so I reckon if it's good enough for them...
I used wood stocked M14's never had a problem with weather affecting the shot any more than altitude or temp on the bullet itself. If you travel to any difference in altitude and climate with any rifle, your poi is going to change. That's why we re-zeroed when we got in-country. Even M4's which have no wood in them.
With that being said. Wood has a notorious internet reputation for swelling or shrinking and just not being all that great for stocks...funny how it's still in use by them fancy high dollar rifles them guys who can afford to fly all over the world and shoot things with buy though...
Laminates are supposedly better than wood, cause the glue helps seal out some moisture... Too bad it's the insides that are glued and the ends (which will always soak up the most moisture) and outsides don't have any glue on them.
Synthetics are supposedly the most stable...least likely to shift your poi...yet they're still glass and pilliar bedded, you still have to check the torque on the action bolts, and they still have to be re-zeroed if you change location/temp/etc...(admittedly, this is because of the outside forces on the bullet, not the stock)
Wood stocks can require more care, depending on their finish, but they can take just as much abuse. Synthetic stocks are all the rage because it's cheaper to make a synthetic stock, easier to injection mold than it is to carve and finish wood, and this means more profit for the rifle makers selling them. Some good advertising and propaganda and the public believes it too.
Don't get me wrong, there's nothing wrong with a GOOD synthetic stock. I own some and like them. But they're not cheap. You end up paying just as much for a good synthetic stock as you would a decent wood one.
But to answer your question, no. All things being equal a wood stock will no more affect a rifle's accuracy than a synthetic. The inletting, bedding, and fit of the stock will. But if you took a rifle action and barrel that shot well in a synthetic stock that was inletted, bedded properly*, and torqued correctly; moved it to a wooden stock that was also inletted and bedded properly* and applied the same torque the accuracy should be the same.
* when I say properly bedded I mean pillar bedded as well. Preferably with pillars just shy of touching the action and a skin of bedding over them...