Is there a pressure point built into your rifle stock? Some of the lighter sporter barrels need them. Could try using credit cards slipped between your barrel and stock to tune it. I hear it tunes better if you're using your wife's credit cards....
Have you checked your action screws to make sure they're not loose? If they're not torqued right that could lead to bad groups. One way to check is (if you don't have an in/lb torque wrench) is to have the front one just barely tightened and gradually tighten it up as you group. Can do the same with the rear. Good beding job never hurt either.