Don't tell deer it isn't enough, my daughter has been filling my freezer with one since she was 9 years old. She only graduated to a .300 Savage this last season because she wants to hunt elk in 2015. Plenty of good bullets out there to load or buy factory that makes the .223 an adequate deer rifle.
The .223 wasn't designed to wound, it was designed to allow a soldier to carry a larger basic load. Try packing 210 rounds of .308 or .30-06 on a 20 mile road march, and you'll understand why the 5.56 is being used. It's all about bringing the most ammunition to the battlefield.
It'll depend on where you are hunting deer too, they're not all the same size. I remember thinking the deer in AZ were like little dogs compared to the ones in NY