With a proper pocket holster, its just as safe as anywhere else. I have carried a Ruger LCP in my back pocket a few times with no problem. It just felt a little uncomfortable to me sitting on it.
But I have a friend who always....7 days a week...carries his like that.