Every accident I can remember hearing of took place at the "truck", or crossing a fence of falling down while walking or at the house. Hunting is safe, but walking in dark woods with a loaded hair triggered rifle is not safe IMHO.
I agree that hunting is safe... and that I do most of my hunting while walking around the woods with a loaded rifle. If it's dark then yeah, I'd probably be unloaded. I don't have a rifle in my hands when I cross a fence so that's not an issue.
The safety I value most is the one between my ears, and has nothing to do with whatever the rifle has.