Weather does and should make a difference. Try quick drawing from an IWB when you got a parka and a heavy sweater on.
Pocket carry in a heavy winter coat just makes sense.
Having worked and lived in several "bad parts of town" in several cities, I just don't see the threat as much greater than the regular parts of town.