It depends on where you're hunting. If you don't have to walk far to get your deer, a heavier rifle is fine. But frankly, no one wants to tote a heavy rifle around for miles. Keep in mind you need to carry your deer out, too. That extra weight is something you just don't need.
That said, I have hunted with a 10-pound rig before. I had my reasons, though. At the time it was my only caliber-appropriate rifle, and I have unsteady feet so I needed to shoot sitting or prone, hence a bipod. Since then, I have added lighter options to my gear kit. Hiking in rough terrain with a heavy rifle was NOT fun.