Although I live in the suburbs outside Philly, I end up going into the city about once or twice a month. Having just turned 21, I don't yet own a handgun let alone have CC. However, I usually feel pretty safe making these trips.
I think it all comes down to situational awareness and making sure you don't get caught in a bad place. Much of the crime in Philly (and I'm sure in most other large cities) takes place in the poor neighborhoods and later at night. Don't be in these areas, and if you have to be there, stay alert.
I also keep to the more "public/crowded" areas with more people as you're more likely to get mugged/robbed by yourself since you're an easier target.
Bottom line? If you can carry, then do so, but either way stay alert and stick to "safe" areas.
Just my two cents