The .454 is a fine round for deer. If you get a .460, you have a .460 with very good range AND a .454 if you want a little less as well as a .45 Colt for plinking with either. Folks have argued over what is enough or too much gun for deer for years. It is interesting that a person with a .45 Colt and off the shelf Federal .45 Colt ammo (not Buffalo Bore big buck stuff costing about a $1.75 a round) that has less than 375 ft/lbs of energy will tell a guy carrying a .327 magnum with Federal factory loads that has over 500 ft/lbs of energy that the .327 is too little gun!
If you handload, you have more options with any handgun. Buy and shoot what you want as confidence in what you have usually means knowing the gun limitations and better shot placement which means more than the caliber.