A bullet...any bullet...should not "keyhole" like that regardless of range.
Bullets do in fact "key hole".
I have many (and have many) 168 SMKs key hole at 1000 yards. That being the same load that shoots quite well at 600.
I've had 535 Gr bullets out of a 45-70 Keyhole at 800, 900, & 1000 yards do to the bullet being cast too hard and too much velocity.
The same bullet using pure lead, kept to 1100-1200 FPS being quite accurate.
All bullets are different, and I can assure you it would not be unreasonable for a 38 to become unstable at 500 yards. I haven't shot 38s at that distance but I have seen some keyhole at 100 which other wise would be quite accurate at 50.
I've done some CSI work (still certified to teach it) and I can tell you I've seen some bullets do some weird, unexplainable things.
One thing you can take to the bank ref: Firearm Investigation, is no two bullets from the same gun fired into the same media will react the same way.
The same with the bullet in flight. Take the 168s mentioned above. I've shot several at 1000 yard matches from M14s. You can get some that shoots like you want and then you'll get keyholes, or flyers.
Out of a 22" barrel with a MV at 2550, its subsonic at 1000. On the 308 you need to remain super sonic at 1000 to have any success.
Then as to the 45-70 black powder load, if you use hard cast bullets and try to put them to 1500+ fps it will become unstable, where as using soft lead, limiting the velocity to less then 1200 fps, they can be quite accurate.
The whys and why fores are difficult to explain, and very.
To understand why some bullets do certain things study:
Understanding Firearm Ballistics by Robert A. Rinker
Gen J. Hatchers writings
and Bryan Litz of Berger Bullets has written several books on the subject.
Check out Litz's latest book "Modern Advancements in Long Range Shooting"
He has a good section on How Stability affects bullet flight.
Just understand nothing about bullets and bullet flight is sure and certain.