I think Thump_rrr hit it on the head. A big differentiator is weight.
When you're hunting, you're going to be carrying the rifle much more than shooting it, and you may be climbing with it, backpacking with it, and/or covering a lot of ground with it. The weight of the rifle is critical, and many hunters use bolt actions that are relatively lightweight...barrels, polymer stocks, etc.
Long-Range Precision Rifles tend to have barrels that are designed to minimize fluctuation and maximize consistency, they're thicker and heavier, they may have bipods and more complex optics, and there may be many other design aspects that tend to add weight to the weapon.
Any rifle can possibly serve either purpose - true. But for hunting deer and elk, especially at high altitudes in the Colorado Rockies where I hunted, a lighter rifle meant I could go further with it.
Treat everyone you meet with dignity and respect....but have a plan to kill them just in case.