Much as I love the M1A...
I would not choose it for hunting (deer) unless I was using handloads tailored to the gun. And it is not the rifle, it is the ammo. The rifles are certainly powerful enough, and accurate enough.
But the M14/M1A is an improved Garand design, and is intended to run on GI ammo. GI 7.62 Nato (.308 Win) is loaded to the same velocity as the old GI .30-06 load, a 150gr @ 2750+/-fps. And it is loaded to fall within the correct pressure curve for the rifle. The rifles also do well with the heavier (172gr) target bullets at appropriate velocity.
The problem is that commercial .308 Winchester is loaded hotter. Sometimes nearly 200fps hotter than GI ammo. And that can put a strain on the rifle. Also the pressure curve may not be optimum. If far enough out of the range the rifle uses, it can cause damage to the gun. The odds are high that the M1A will stand the slight overstrain without damage, but why risk it?
Heavy loads, or loads at the wrong end of the pressure curve are know to have bent operating rods in M1 Garands, and have the potential to do the same to the M1A.
If hunting with the M1A, I would load ammo to GI specs, except substituting a soft point hunting bullet for the military FMJ. Or, if limited to commercial ammo, just turn the spindle valve off, making the gun a stright pull bolt action with spring closure. This simple step will avoid any risk to the gun from ammo with an out of range pressure curve.
I have also heard (but cannot verify) that some 180gr RN bullets have been known to have some lead shaved from the nose during the feeding cycle, which can build up causing jams. Something to be aware of if you use this bullet weight and nose profile. I have never used it in my M1A, so I cannot say if it is true or not, only that the rumor exists.
Using tailored ammo, or used as a manual repeater, I would not hesitate to use the M1A for big game hunting with a magazine size that meets game laws (usually less than 10 rnds for hunting). 5 rnd mags are available and generally work fine, except they are a bit awkward to remove because there isn't much sticking out to grab hold of.
if the M1A was the only rifle I had, I would use it, under the above noted conditions. Otherwise I would use a lighter rifle, one purpose built for hunting, to save weight.
All else being equal (and it almost never is) bigger bullets tend to work better.