I have heard it said that a gun is like a parachute. You will probably never need one, but if you do, your life will depend on it.
Keeping it in a lock box/safe slows down your response time. Do you want to slow it further by accessing an unloaded gun?
For the record, I am not knocking the safe, just stating a fact. For my money, if it is in a safe, it is safe. I don't need to unload it to make it "more safe".