That pin is slotted and looks like a screw, but it is not. The slot was intended to be used during assembly to spread the head and so retain the pin. In practice this was not always done; the pin is often retained by the stock and can fall out when the stock is removed. When it does, the ejector falls out and you pick the brass out by hand, like you say.
Jim