Depends on your state laws. Signs saying carrying of weapons not allowed don't mean squat in my state. Only thing is if you don't leave when asked your trespassing and then I'm sure the city attorney will find a reason to add a weapons charge in too.
In Kentucky those signs have no weight of law behind them either. There are of course places that you can't carry, like a court house for example.
Laws very by state so I'd suggest using www.handgunlaw.us
as a great resource.