Do your own tests.
Personally I step left. Any right handed person coming at me with a weapon will have to track me to their right. It's much easier for them to swing a gun of knife across their body to their left that it is to swing it to their right. That requires them to turn their entire body.
I know it sounds like leaning into a punch but unless you can create safety by moving backwards you're better off by closing in where they lose their range of motion. This applies ONLY to an armed opponent!
If the threat is using unarmed tactics or a strike force weapon then you do just the opposite and move backwards and away.