I think you need to get a new dog ... one that will bark big and deep when someone jiggles the door. Yes the guy knows where you live. Whatever attracted him to you has not gone away. He might pick another time to return visit for whatever he had in mind. Be careful.
When the number of people in institutions reaches 51%, we change sides.