I agree with Spats McGee, although I don't wear suits anymore (now that I'm retired). And since I only carry a gun when I legally can, it's mostly on my trips to Arizona and Nevada.
One generally needs to "dress around the gun." When someone begins to carry a gun conceal, he will often need to make some modification to his wardrobe.
I've actually personally had to make only minor adjustments. I've pretty much always dressed in khakis, a button-up shirt and either a jacket or light vest (I like having the pockets).
"It is long been a principle of ours that one is no more armed because he has possession of a firearm than he is a musician because he owns a piano. There is no point in having a gun if you are not capable of using it skillfully." -- Jeff Cooper