jimbob86, really baggy is subjective. I have no trouble concealing a 1911 IWB, with either a shirt or sweatshirt loose over the gun (and a t-shirt between myself and the gun), or under a jacket.
Having been in the Army much of my formative years, I had it ingrained in me that T-shirts are tucked in .... that's how I dress around the gun as well. I did have to get larger pants and longer shirts ...... and summer time I usually have to lose the cover garment and untuck the t-shirt (though if I'm going to be out in the sun, I'll keep a long sleeve work shirt on to keep from getting burned .....
One thing about the 5" 1911 IWB- though it is large and heavy, it is very flat, and the grip is generally smaller than most double stacks ..... even for a large gun, it carries better than many guns that are much lighter and smaller, because it is thinner.
When I first started Carrying (5" steel framed 1911a1 in .45ACP), many people with more experience told me I'd eventually get a smaller gun ..... I told them stuff like "It's supposed to be comforting, not comfortable." ..... As soon as I could afford it, I got a smaller 1911 in a caliber that was more comfortable to Carry and cheaper to feed for practice (and now I reload 9mm).