I think of it like this. You pay for a hunting license and gas to go. Sure, any caliber could hypothetically kill a deer. But why would you chance it. You can buy a single shot handi rifle or a rossi in a more suitable deer caliber.
A hit with a .22 is better than a miss with a .44