I'm not a hunter. I know people who have taken Sitka Black Tails with 9mm and .38 SPL. I had a friend would would hunt exclusively with his S&W model 627 loaded with 180gr flat points. He was quite successful, even out to 100 yards with iron sights. Heck, where I used to live, .223 was an adequate round to take one of these small deer.
Having said that...if you want to challenge yourself with pistol hunting, get yourself an adequate pistol for the job. At least 44 Mag, or 10mm with the proper ammo. Leave Mr. 9mm (as
Jules calls it) at home.