My suggestion is check out the wide array of products on the web... I know that is very general, but only you know what will work. Order from a company with a no BS return policy and a wide array of products like
www.action-direct.com I've used them and they are good at returns.
With your description of yourself, my thoughts are to think about realistically where can you carry something. These thoughts come to mind:
1) Can you do a fanny pack? It's not realisitic or practical for everyone, but it may work for you.
2) Do you want the gun on your body, or is off body carry like in a dayplanner okay?
3) Carrying a 5" XD CCW is not always realistic given your restrictions. Can you get the 3" XD or a smaller gun for IWB carry?
4) If on the body is your only method and that is your only gun, how about a shoulder rig from Bicanchi (Miami classic for instance) or Ross or Galco of Fobus?
5) I use a rid that works very well for a 3", 4", or 5" XD, depending on the length and weight/thickness of the shirt. Works best for longer button down shirts of alittle heavier weight, sometimes with an undershirt. Its a Ross leather sleeve that slides on the outside of my belt and the gun sits on the outside of my jeans, covered only by my shirt. Rarely prints and I'm just cautious of certain leaning over motions that may cause printing. Works well for me.
6) If you can get the 3" XD, Galco skyops makes a great IWB holster for the 3" or 4" XD which allows you to tuck a shirt in over the gun. Many other makers make a "shirt tucker" similar to this. Be forewarned that while the Skyops holster works well, it requires a thin dress belt. Others work with thicker belts.
Good luck