I can't believe someone who says they've hunted any decent amount and a deer has never not dropped in sight. I've shot a buck straight through the center of the heart with an arrow, and had it run 200 yards. I shot a buck with a 325gr FTX out of a muzzleloader at 10ft, through the front, destroyed virtually every organ in the animal before stopping in the back ham. The deer covered 150 yards just from the shock of the shot so close. Not a week ago a shot a doe through both lungs and took out the top two chambers of the heart with a 44Mag carbine, and she covered 100 yards.
Now shots at >150yds, where they don't hear the shot so close, those are virtually always bang-flops.
And a deer going 3/4 mile after being shot with a 300WM, someone needs to either choose a better shot angle, a more appropriate bullet, or both. I hunt with one and the only animal it hasn't been massively overkill on is an elk at 700 yards.
As for lightening a rifle, IMHO you are probably cheaper to start with one that is already made to be light and choose light mounts and a small, lights scope. If you won't be hunting over 400 yards you can get away with pretty minimal optics.