I don't see any problem with allowing 223 for deer.
As my father used to say, it's the indian and not the arrow.
I've taken 300+ lb. hogs with my AR carbine without problem but I had to be patient to wait for the perfect opportunity. Most bad shot on a game with a 223 would not have ended any better if the shooter used larger caliber/cartridge.
You can't make up for bad shot with "good" ammo.