a hunting rifle will never be used in the field with a warm barrel...
Define "warm".
People hunt in conditions where the barrel (and everything else) can be over 100 degrees.
My guess is if it takes more than 2 shots then the barrel temp changes your POI
There is more involved than just the barrel temperature. How fast a barrel heats up, and how much depends on a number of factors. Some powders heat guns noticeably "faster" than others. High pressure cartridges can be different than lower pressure rounds. Physical configuration of the barrel itself (thin light weight sporter vs havy varmint, etc) to name just some. And, of course how close together your shots are in time.
It is a long observed fact that first shot from a clean barrel can have a different POI than follow on shots. Sometimes the difference is a couple inches.
I've known guys who will not clean a rifle bore during hunting season, unless they HAVE to. And, if they do, they fire a couple of fouling shots THEN check their zero to be sure.
For a hunting rifle, I think the important thing is not the size of a 5 or 10 shot group, its having that FIRST shot hit where the sights are. Rapid follow up shots should be close enough to stay within the "minute of deer" or whatever you are hunting.
Look at it this way, if you were (or are) using a single shot rifle, how would you sight it in? Generally speaking, while there can be a significant difference between that first shot from a clean bore, and the next shot from a now fouled bore, there is usually much less difference between shots after the barrel is "fouled".
I would foul the bore with a shot or two, (which removes any traces of oil in the bore) then sight in with enough time between shots that the barrel doesn't noticeably heat up. Once your bullet strikes the point of aim, you're good to go, with a reasonable expectation that your first shot in the field will be on target (assuming you do your part, properly)