Being an old timer I still like map & compass navigating, but I have been dragged (kicking & screaming) into the late 20th Century by getting a GPS.
FWIW here are my observations from the POV of a GPS newbie.
YES they are very good at getting you back to somewhere you've been to with them.
(What you do is "save a Way-point" (location) when you are there.) Really easy you just push a couple of buttons, say "YES!" & enter a name so you can tell it from all the other way-points you've saved.
You can use this saved location in several ways.
You can just have the GPS tell you the saved way-point is (X) miles in (Y) direction. (The easiest but least useful method).
You can navigate from way-point to way-point in direct lines. This lets you, for example get from your tree stand to a different blind that you've marked, but have never been to from here.
The GPS will create "steps" to each point telling you each point in turn is (X) miles in (Y) direction from the next one.
You can have the GPS actually record your trail exactly as you walked it.
The trail is saved & named, just like a way-point, you then follow a "breadcrumb trail" step-by step in either direction.
Tricks.
Start collecting way-points as soon as you get the little bugger. Way-points are the key to a good use fro a GPS.
GPS is not as good at Getting to there from here as it is to coming back to here from somewhere else.
GPS is not perfect & there are things that can mess it up, most of the problem being fixable by just moving a few yards till it gets sorted out.
You will love it when you get past the confusing new terminology about coordinate systems & NAD's & so on.