It's like a baby being born - it will come out. It wants to come out.
First of all, make sure you're pushing it out the right way - from the right side.
Use a wooden dowel or something plastic that's about the same size, line it up, depress the spring tip, hit the dowel with a hammer.
Try backing the screw on the left side out.
You don't want to beat the crap out of it, but you can't really baby it, either. Pretend you're in the middle of a heated battle and you want to take out the empty cylinder and replace it with a loaded one...you'll get that wedge out somehow.