No question that the right answer is get whatever gun you WILL carry - 24/7. The problem that most people make is that they think with their "manly parts" when purchasing a carry gun, rather than their brain. This results in purchasing the biggest cannon their belt will hold up. They carry it for about a week and then it sits for three months until they trade it in. I would look hard at the smaller 9mm pistols. The Kahr that is mentioned is a good one. You have mentioned that you "fitted" the LC9. How about the new Sig P290 or the Taurus 709 Slimline. If those are all too small, then I would look towards some of the sub-compact models such as the Beretta Px4, the Springfield XD or the Ruger SR9c. They are all very reliable handguns.
Let me know if I can further assist you.