nocturnus, it sounds that maybe she was already up close and personal talking to this man by the time you were on the phone....like if I or my wife is talking to the ups guy needing a signature as an example.
At this point if what I just stated above is true, I would just ask the man what he wanted. when your wife told you, was there a reason for him needing to speak with you? there should've been if he needed to speak with you or if your wife was stumped, a little perturbed, and she sectioned the situation to you....
At that point he already had access, so him hanging up is just making a situation worse if he is harboring mal-intent IS ALL I AM GETTING AT. I am not saying you did anything wrong, but thank God your wife answered when you called back. The other thing is, you need to know his purpose to also rule out the possibility that in fact this guy was actually an AT&T employee:
I was scared that this guy was not who he said he was