I'd like to shoot something as close as possible to what was commonly used in the late 19th century, terms of caliber/chambering.
If you want historical accuracy as close as you can get without paying the price for an original get a Uberti 73 in 44-40, 38-40 or 32-20. The Rossi 92 is a good gun for really late in the century. If you want something with a more powerful caliber look around for a good used Winchester 94 top eject in 30-30. Forget about the modern Henry, they're nothing like what was used back in the day. The Marlin 94 is a decent gun but it comes in modern calibers.