Its not like there is one best choice.You will be fine if you select any of a number of appropriate choices.
Most of the bullet manufacturers give a velocity range their hunting bullets are designed to perform within.
The thing to understand is a bullet that will perform perfectly from a 300 Mega-Mag at 3200 fps striking a a moose at 100 yds likely will not work so well from a more moderate 30-06 at 2800 fps impacting the target at 600 yd velocities.
So,you are shooting a deer,not a moose.Extreme penetration is not required.
Lets guess you can get 2950 with a 165 gr bullet from your 28 in bbl.
A Nosler Ballistic tip will be traveling at about 1890 fps @ 600 yds(based on sea level)
It will have about 2200 fps at 400 yds.
These are roughly the velocities a 30-30 typically works with.They will kill a deer just fine,but you do not need a tough ,thick jacket,bonded bullet.
While there are other good choices,a 165 Nosler Ballistic Tip is one good choice.
Probably the most important info you can ask your bullet manufacturer(call them!) is "What is the lowest velocity I can expect reliable expansion ?"
If you can,set out some form of wind indicators,like a fishing bobber on a string.
At those ranges,a little wind drift or 40 yds error in ranging can result in a gut shot,or a 3 legged deer.
I am not suggesting you cannot or should not do it.
But if the range is 537 yds,you can see just a touch of mirage and feel a little bit of breeze on your cheek,in 10 seconds or so,what is your hold?
What I'm wondering...It might be that you are a world class Palma shooter or former military sniper who is very competent at hitting targets at long ranges under hunting conditions,and you just aren't too sure about hunting bullet performance.Thats cool.Your question on bullet selection might make me wonder.
I hope if you are going to shoot deer at those ranges you develop the technical skills to give thedeer a quick,clean end.