Small, light bullets are the most difficult to pull with an inertia puller, as they have the least inertia. An "expert" with the inertia puller can do it.
If the .223 is GI stuff then seating the bullets slightly deeper to break the seal is often needed.
I went years with the RCBS collet puller, with often less than satisfactory results. Many times it was impossible to hand tighten the collet enough, and it would slip off. And, of course, forget about pulling lead bullets in any condition other than scrap for the lead pot.
I retired the collet puller when I got a Lyman "orange hammer". I have pulled .223 and .22-250 with it, as well a many larger bullets.
The trick, for those who aren't experts, is not how hard you swing it. The trick is the sudden stop. Many surfaces that are very hard to our bodies are "sub-optimal" for the hammer puller. Concrete is common, and seems very hard, but there is enough "give" to it when the hammer strikes to actually cushion the blow to a degree.
What I have found works best is the hard cast iron top of my wood stove. (COLD STOVE, of course). A few quick "snaps" is usually all it takes, though the .22s might need 5 or six.
Up to you, but the hammer works BETTER, for me, than the collet. Plus, the hammer also pulls lead bullets, undamaged!!