It's the "lock the slide back" part that's so hard. I use my stronger arm (right) to pull it back, but by the time I have it lined up to try and get the slide button up, the slide goes forward again. Takes me about five minutes to get the slide lock in place (and then I'm beat! lol) Maybe this is just one of those times where it pays to be a guy. Unless your wives have no trouble with it, and then that would just make me feel like a wimp.
I don't know about the XDM, but my XD's recoil spring assembly is pretty stout to pull back and latch. It's probably the hardest of any of my autoloaders to rack.
Try the empty magazine method, so you're not having to try to work the slide latch at the same time.
"As was the man of dust, so also are those who are of the dust, and as is the man of heaven, so also are those who are of heaven. Just as we have borne the image of the man of dust, we shall also bear the image of the man of heaven. "