Parts of downtown Orlando are trash but others aren't. If you stay inside of the core where the highrises are you're OK. Other parts are definitely "at your own risk" when traveling. I park across from OPD headquarters all the time and never have a problem.
Stay out of Pine Hills, period. My mother in law had a home there in the 60s when it was the premier neighborhood in Orlando for the white collar crowd. Most of those homes are still there but it's turned into a terrible place. I lived there myself through the 90s and that's when it really started to turn. I moved out in 2003, I lived at the far edge, really it was the Clarcona/Rosemont area. After several shootings we moved.
I have lived in Maitland since then and even here it's creeping in.