Think of it this way... If you have two objects of the same momentum, and one applies a force over a short distance (the bullet) and one applies a force over a longer distance (the rifle against your shoulder), the work done must be equal in each case. To compensate, the force for the shorter distance has to be higher than the longer distance.
And remember, its time rate of change of momentum... The rate of change is higher for a bullet than it is a rifle...
Interesting, since we are talking about specific terms, I would have assumed that the bullet applies its momentum over a longer distance (penetration) than the rifle (recoil distance). I don't know of many rifles that provide 10-14" of recoil to the shooter, though many rounds provide 10-14" of penetration.