.380 is definitely a better stopper than .32
This is not scientific, but my opinion based upon having shot both calibers and reading widely.
Shot placement is key, as with any handgun, but due to the marginal power, it is even more important here.
I would carry modern hollowpoints (the Gold Sabers sound good), assuming my pistol was reliable with them, otherwise FMJ.
I think .380 and .38 spl are about the minimum for effective self-defense, unless other factors make a smaller caliber pistol necessary (i.e., the need for a deep-cover "mouse gun").
Interestingly, I think the .32 is a very efficient round, in that it gives the most stopping power for the least "bang", so if you're a wanna-be spy or assassin, it might be a better choice.
Again, I think hollowpoints are the way to go, but a case can be made for the exotics here, like Glasers.
Overall, a pretty specialized niche weapon.