There is a big difference in 45-70 loads. If you stick to traditional loads at old blackpowder velocities the recoil is resonable, around 24-25 ft lbs. About the same as heavy 30-06 or milder 300 mag recoil. If you you use some of the really hot loads you are looking at near 50 ft lbs of recoil. This is actually more than a 375 H&H mag. These numbers were figured with a 7.5 lb gun. Heavier or lighter guns would make a differene, but I kept the weights the same to try to get a fair comparison.
Just for reference a 12 ga slug will be run somewhere between 28-30 ft lbs of recoil from a 7.5 lb gun. You are really getting beat up with a 5-6 lb gun shooting slugs.
Another thing to consider is stock shape. All of the H&R single shots have poor stock designs that seem to magnify the effects of recoil. A bolt gun with a modern stock design will feel as if it has much less recoil.