It has already been said, it's YOUR gun and YOUR life, carry how you are comfortable.
My only contribution is this, what about something like a Springfield XD line pistol for chambered carry. No manual safety to forget, but has both a grip safety and a longer "not quite double action" pull with a firing pin block for added drop safety. You really really really have to mean to do it, to make one of those go off. That is also true of many other guns, but I just wanted to point it out if it was an option to you.