I carry a service model XD in a IWB at the 6 oclock position. I never had to get a larger pants size for this carry position. Sitting isn't to uncomforable unless it's for a very long time. I live in AL, so I'm in the same heat situation u are in. I've adjusted to wearing a "wife beater" or light t-shirt under my main shirt for the day. I carry the USP full size which is a little bigger in a OWB @ 3 o'clock with a untucked button up shirt to cover it. If Im going to be driving or sitting alot that day i wear it in a crossdraw position. The XD could be carried the same way if you wear appropriate cover shirt length wise. I am 6'1" 183lbs if this helps you figure anything out. The XD is a gun thats easy to carry for CCW in my experience. I like the no manual safety thing.
If you know the enemy and know yourself you need not fear the results of a hundred battles. - Sun Tzu