About the fastest, loudest thing I shoot these days is the 40 S&W with hot 165's. I've seen HST's drive across 13-14" of deer shoulder and it piled up right there. The .40 has replaced the 357 as my 'medium bore' handgun round.
The best deer killer Ive ever used is the 45 Colt with Sierra's 240 JHC with enough W296 to scoot it along at about 1300 fps. That is definitely loud but it shoots flat and it sure works. I have also killed them with 44/45 cast SWC's at 1000 fps or so, and they're equally effective for 'spine at the shoulder' shots. Either will work if well applied.
Like you, I have never seen anything resembling rifle wounds from a handgun round.