When I'm carrying concealed, my rule is that I will not draw my gun unless I'm taking it off for the day or there's a pressing need to draw. It's just so much simpler that way.
There are too many things that can go wrong when you pull out a loaded gun someplace other than a shooting range. For one thing, in any kind of a social setting, it's usually pretty difficult to exercise muzzle control--finding a safe place to point the muzzle isn't always easy.
If it's really necessary, then it's worth the risk. If it's not necessary then why do it?