Kind of hard to answer that one, how hot are the lighter bullet loads. F=MA so you have both a change in mass and a different acceleration. If all were equal except bullet weight the heavier bullet will have more felt recoil. In my Accurate #7 loads I use a couple tenths over min and get reliable cycling in the action. They don't seem like stout loads to me but I'm not that sensitive to recoil on the hand or shoulder if shooting a rifle.
Gun control is hitting what you aim at..