I have a variety of handguns. I tend to think a .38/.32mag snubbie is the most practical self-defense gun for me - but it too is an amalgam of compromises. If it's ever used to shoot someone in a self defense situation, it will likely be done so at a range of about 7ft.or less. Yet, at the range, I see all these suburban commandoes in states of hypervigilance shooting .40S&W's out to 25yds. In the back of my mind, I note that during WW2, Churchill's bodyguard used a .32 and that U.S.Generals were issued Colt .32's.
One of the joys of plinking - is the lack of recoil. The other joy in plinking - is to hit the target. If one has a lot of recoil to deal with - well, half the fun is gone. I think a lot of the folks - especially new shooters who shoot big calibers - would probably enjoy things a lot more with a smaller caliber. I also think folks need to study more about self-defense itself than 'caliber.' Better to master a .25acp than to bungle a .45.