Best idea is to give him what he wants and hope he doesn't shoot. The only way drawing might be a good idea is if he's likely to shoot you, or you can draw and shoot your handgun from a concealed holster in about a quarter second or faster. Even then, he might manage to get a shot off before the hit registers in his brain.
If you think he wants to do you harm, any manner of distracting BG to pull your gun is probably a good idea. If he just wants your money, give it to him and hope he's satisfied with it.