Well, I for one, have experienced no physical problems related to shooting magnums nor heavy loaded .44 Special or .45 Colt ammunition.
I first experimented with heavy .44 Special loads around 1953 or 1954, and have shot a full range of hard kicking stuff up to the .45-70. This amounts to some quarter million rounds fired over nearly sixty years. Maybe its the way I shoot or my grip, but never felt any damage except to my hand when using rubber or ill-fitting grips.
I do have a ringing in my ears from lack of ear protection, but that came as a result of 90mm tank guns and/or 106 mm recoilless rifles.
Don't get me wrong, I'm not a macho man by any means, but I believe those who have problems have it because of diet, life style, or some cause from natural aging.