Never had it happen in 12+ years. So many people carry cell phones in a holster that anyone without a bulge is suspicious. My wife knows that I carry and is often reassured to know that I'm armed when we are out and about. I don't really want her touching the gun though. If she comes in for a hug or kiss, I preemptively put my arms around her waist. That forces her to put her arms around my neck. If I'm in a large crowd and it's likely to get bumped, I'll tuck my elbow in close to my side and guard it with my elbow. I also try real hard to not be in crowds that are that tight.
To a much greater extent than most mechanical devices, firearms are terribly unforgiving of any overconfidence, complacency or negligence.