I'm not really sure why so many people favor "small" rifles for deer. I know I am stealing a line here, but all other things being equal, bigger bullets usually work better
As long as you can still shoot a slightly larger rifle equally well, I would not recomment a 5.56 or 6mm or perhaps even 6.5mm rifle for deer. (though I know they
can be used effectively I believe moderate 7mms and 30s have an advantage) Perhaps if you have some reason, like recoil affecting your back problems, or only having a budget for one rifle and wanting to use it to hunt small varmints as well as deer, then you could compromise and make due with a "smaller" rifle.
By the way, why rule out .308 just because you are thinking of getting a target rifle in that caliber later? Having a deer rifle in the same caliber
MAY allow you to use bullets with the same down range ballistics and save you a bit of memorization when it comes to long shots, and if you ever do get into handloading, you could most likely use the same dies and save some cash and/or clutter.
Anyway, of your first list of calibers, I would go with 7mm-08 for a nice light recoiling, moderately long range rifle with decent knock down power on deer. Loaded with the right bullets, it is a decent little round. But the .270 has a long history of getting the job done nicely as well and might be easier to find ammo for if you are in a pinch and need to get your hands on some with very short notice.