A target rifle is built heavy---heavy barrel, over-sized stock, and so on. It's all about providing the most steady platform to deliver the tightest group on target at prescribed ranges. Now... That being said, accuracy counts big-time with hunting---yes, is all about shot placement. BUT, do you really want to pack that 15-pound percision target rifle that you just spent 4K to build with $1,500 worth of nikon or swarovski optics up and down the mountains in less-than-fun weather/terrain/etc. to get your deer? Didn't think so. Yes, percision counts in hunting. But it's a beast of a different nature. Use the tools that get the job done and you'll do just fine.
My son shoots incredible sub-MOA groups with all my target rifles. This year he's using my Mosin M-91 when we go for elk (a $200 battle-axe that happens to be tight-grouping with his trigger-pull) and I have no doubt there will be meat in the freezer from both of us.