Quick answer: When the trigger is engaging, the gun is moving.
Long answer: There are 5 (or 6, depending on which school of thought you follow) shooting fundamentals. Most of us are not born with the ability to be precision shooters. Shooting is far more mental than it is physical.
The fundamentals are:
- Position
- Grip
- Breath control
- Trigger control
- Sight alignment
- Follow thru
The 1st three and #6 sets up a good platform for #4 & #5, which are the two most critical for accurate shooting. If the gun does not move while the trigger is being pressed, and up to the time the bullet leaves the barrel, and assuming the gun is reliable and functional, the bullet will hit where the gun is aimed.
Without seeing you shoot, I would be hard pressed to accurately diagnose what you are doing incorrectly. Based on the information you've presented, and based on my own thousands of hours shooting and hundreds of hours instructing others to shoot, I would guess that you are
1. Flinching
2. Flinching
3. Maybe flinching.
4. Right hand applying too much (grip) pressure to the gun
Flinching is a clear indication of lack of concentration on keeping the front sight aligned. Flinching shows that you are thinking about controlling the recoil that is about to happen. The new shooter cannot properly control recoil anymore than the new bullrider can stay on the bull. The goal is to be able to not worry about recoil too much, and "ride it out" (like correcting for a loss of traction going around a corner on a motor vehicle).