When using a handgun at longer distances remember in handgun calibers the trajectory is like a rainbow. Bullet drop is significant when going beyond 50 yards. Best to select the range you're proficient at and find a mulie at that distance. Several ways to judge this, use a range finder or know what the deer will look like in relationship to the crosshairs. Practice at this range until you can consistantly get about 6-8" groups.
Elmer Kieth took game with a .44mag at 600 yards. I'm no Elmer Kieth, but I have taken Antelope and Mulies out to 200 yards with a .44 mag. It's all about your skill level and proficientcy. My favorites are the Desert Eagle and Performance Center .44.