I will make a few assumptions since you didn't mention it. I assume you load the mag with the number of rounds you want to shoot in a single group and shoot. Therefore the first round is fed by operating the bolt by hand and the rest are self fed by the rifle's action. Your first shot is the one that hits outside the group.
So once you release the bolt on your first round, push hard on the closed bolt handle to try to copy the force of the action when it slams forward to seat each additional cartridge.
Or you can load one round at a time into the mag and release the bolt by hand for each shot.
Another trick you can try is to push a patch with a couple drops of oil through the bore before your first shot.
If none of the nonsense above makes a difference, ask someone with a lot of experience making minis shoot:
If nothing else watch his videos on mini trigger jobs and bedding. I had him work over my mini-30 trigger. It was well worth the money.