It's the shots I didn't take that I remember the most. Not for the sense of loss but for the sense of wonder that I made the decision to not take the shot. For me, the hunt is over once the crosshairs settle on the animal. I could just as easily be clicking a shutter as tripping a sear.
That said, there is all kinds of hunting. We've got a rule (for example) on our lease that all feral hogs are to be shot on sight, even if it ruins your hunting for the day. I don't consider shooting hogs to be hunting, even though hogs may well be the smartest animals in the timber. Hogs are something we deal with, not hunt for sport.