Jerry, I am not exactly sure what your rifle is doing, but what I am understanding is that it groups decent but has flyers?
I have never had a Remington that shot great until I squared the action and re-cut the chamber. Their barrels are usually pretty good barrels. Their machine work and assembly just leave a lot to be desired. Having said that, the symptom I usually see are poor groups or groups moving as the barrel heats up. Flyers are usually some sort of bedding issue or barrel contact issue. Even aluminum block bedded stocks can have issues. I have seen some expensive stocks with aluminum blocks throw shots like crazy until the contact problem they had was solved.