MK8 you have the right idea. Practice, practice, practice, and do it on paper targets so you can see exactly what it happening. Coming from a bird hunting background when you get excited you're reverting to what has always worked, just like anyone would, and shooting like you always have. The issue is that shooting a rifle is completely different than shooting a shotgun. I can relate to your problem because in the late 90's I got to see the exact same thing with my cousin and he missed a fine bull elk on public land because of it.
As for cartridge choice either that you have are going to work fine. I've shot many rounds of 7mm Remington Mag and I don't find it uncomfortable at all. I have also seen many animals killed with the 7mm Remington Mag and all have been just as tasty as those killed with non-mag cartridges. The 7X57 is wonderful cartridge, it has a rich history and has very little recoil. Either should do what you need and I doubt the deer well know the difference.