If you like the XD, I'd definitely suggest checking out the M&P. To me it has superior ergonomics. It also has a lower bore axis, so it even has lower felt recoil.
To me the XDm fits my hand way better than the M&P. The XD is a blocky gun though. If I had to rate it I'd go XD < M&P < XDm. I honestly don't notice much difference in recoil between the XD/XDm and M&P in 9mm, but I think part of that is that the XD/XDm are noticeably heavier guns.