It's all relative
In 2007 Kenneth Shipp, after being involved in a traffic incident, drew and fired a .22 magnum North American Arms SA revolver into the face of Officer Eric Freeman. Sadly, Officer Freeman died of the wound shortly thereafter. Conversely, Officer Massad Ayoob (and famed ballistics and defensive shooting expert) fired five .38 special rounds point blank into the face of a perp...and he lived.
Does caliber matter? In most instances, 'yes', in others, not so much. Ultimately I believe that caliber matters. I carry a .357 magnum most (99% of the time), a .45 ACP .5% of the time and a .22 magnum the other .5% of the time (don't ask, just know that's what's required). I believe that a .357 magnum is the most effective round for stopping a human attack...therefore I carry it most often. On the very rare occasion that I think I'll need more firepower, I carry my full size HK45. On the occasions that I absolutely cannot carry anything more, I carry my mouse gun (which I load with 45 gr. Critical Defense loads from Hornady and would not hesitate in sticking in in the eye of an aggresor).
Caliber is important, but shot placement is paramount. Carry, practice and use that with which you are most comfortable and contribute to the NRA.