I'm not saying you couldn't do the job with a .22 magnum, because I know a whole lot of people who could. I just don't like the rimfire cartridge for self defense. I've had enough of them spit on me to tell myself that I'll never trust rimfire weapons to save my life.
However, that's ME. If you've had good experiences with them, and you buy good quality ammunition, then I'd say go for it. Chances are just pulling it out will diffuse a situation anyway. Personally, if I HAD to choose to use a .22 Magnum for self defense, I'd make it a revolver, so if I did have a mis-fire, I could just pull the trigger on a new cartridge instead of running a clearing drill.
ΜΟΛΩΝ ΛΑΒΕ- Greek:"Come and take them..." Meaning: Here we peaceably stand as armed and free men, willing to defend that peace, and ready to make war upon anyone who threatens that freedom.