Some information that would help in providing guidance:
- what power scope are you using and the cross hairs?
- are you diligent at cleaning all the copper and fouling out of your bore?
- have you checked your bedding to make sure your screws are tight?
- is the barrel free floated or dies it have the contact point at the tip of the forearm?
- what mounts do you use and have you checked the screws to make sure they are tight?
I'm a big Remington 700 fan and the trigger is easily adjusted. I normally set mine at 3 pounds, so you might make a trigger replacement lower on your list of expenditures.
I agree IMR 4350 seems it might a bit slow for your load. I normally try and select a powder that gives a fill of at least 85%. The Dan Newberry OCW
system for tuning your load to your rifle.
Nothing personal, but do you have a baseline that defines "your" accuracy potential? What is the best groups you have shot with your best rifle? Some people are the limitation, not the rifle.