People don't like grip safeties for a number of reasons.
Some talk about one hand manipulation drills, yet they can all be done with an XD/M/S.
People complain about not being able to shoot it without a proper grip, yet I've shot the XD/M/S loose enough where it would almost fly out of my hand. I can grip low enough on the XD/M/S grip that I can't reach the trigger, yet the grip safety is still depressed.
People complain about the grip safety because they don't know it's purpose or the type of gun it's on. The XD/M/S striker is more ready to go (basically single action) than a Glock. The grip safety blocks the sear, unlike a 1911 grip safety that people think does the same function. Basically, who here would carry a cocked and locked 1911 with the thumb safety removed? Who has removed it and carried that way? Anyone?
Reholstering is nice since you can put your thumb on the back of the slide, letting the grip safety do it's thing. You should replace worn out gear, but if some flap, strap, jacket end, draw string or loose holster piece gets trapped in the trigger guard as you push the gun back in the holster, the gun will not discharge.
I've owned 7 XDs, 1 XDM, 1 XDS, 12 1911s and lots of handguns without grip safeties and never hand a problem either way. I like a passive sear block over a manual one, but I prefer some sort of real safety over just some tab in the trigger that is more a drop safety than anything else.