I have a full size M&P40 that is my CCW, plus I have used if often in IDPA competition. Have a bit over 10,000 rounds thru it with no issues that I can remember. A good friend has the XD40 which I have some experience with. The XD seems to soak up the recoil just a tad better. Both pistols are equally accurate.
Of the two XD(m) pistols I have experience with, both are VERY accurate.
I too would have a hard time picking just one. It may come down to how it fits the hand.