Springmom, what you descrined is the perfect combat mindset scenatio. Condition Yellow, state of general alert, observing all of your surroundings. Movement to condition Orange, where a specific threat has been identified, and begin setting your triggers and safeties. Your trigger being if he made a move for you, you were prepared to move to condition Red, use of force. However, your safety was met by his inaction, allowing you to move back to conditon Yellow when approriate.
Kudos to you for not allowing yourself to be "one of the helpless."
Life's tough. But it's tougher when you're stupid.