Although certainly possible to terminate someone with a single punch, how does that work out for the 135 lb woman being assaulted by the 230 lb man? Just not realistic for a majority of the population... In WWII the Japanese Army desperately low on supplies fought with swords and knives in places... It didn't work out very well for them, fist would be much less so...
Not to say some people don't have the talent, skills or luck to win the fight but your average person just isn't going to do it..