Yes. It is called lapping. Do not attempt to polish the sides holding the hammer in your hand, you will round the edges.
Lay your polishing medium; crocus cloth, super fine sandpaper, whatever you are using, flat on a flat surface. Really flat. Rub the flats of the hammer over the flat abrasive surface. Don't try to hold onto the hammer. Just lay a couple of fingers at the center of the hammer and push it around on the lapping surface. Push it in a figure '8' pattern to even the wear. Don't rush, lapping is a time consuming process.