I think that the OP's question contains some excellent analysis.
There are downsides to carrying. You can't, for example, have an accidental discharge if you don't carry a firearm. The OP is simply considering his situation and balancing the upsides and the downsides.
I carry all the time, because it's easier than carrying some of the time. However, the OP's reasoning does impact what I carry - like no extra magazine.