Newton's laws state there will be an equal and opposite reaction. This is a long standing fact.
The reason we don't fall over and die is because that energy is dissipated through the gun's mass and area, the shooter's tissue, and the ground. The energy to move a 150lb human holding a 2lb firearm is much greater than a 230gn chunk of metal. Since the bullet has a small cross section, it will penetrate; whereas the stock distributes the forces over a MUCH wider area.
This is the old "two people of different masses push each other on a ice rink" problem. The lighter one WILL move faster.