You need to eliminate the possible causes one at a time in order to solve your zero shift problem. Start with the scope, then move onto other items. You did not mention the distance that the first shot is from the group of four, nor did you mention the range that these groups were fired at. This information alone is very important to understand the magnitude of the problem.
I trust that this information will help you.
Dardas Cast Bullets