If you keep your hands in your pockets most of the time someone will sucker punch you first. That is the street. They don't just tell you they are gonna rob or hurt you. They slide up and try to get a conversation with you and either go from there or a 'stranger' walks on past and blindsides you.
Yes it's faster to draw with your hand on the gun. It's also faster to throw a fast jab than to draw. For there is more to self defense than guns. Guns are a part of it but there is more.
"The government has confiscated all of our rights and is selling them back to us in the form of permits."