Most people I know have their house keys on the same ring with their car keys, which is a bad idea.
I know none of us has "ever" left the car running and ran into the Quick Mart to get something, or something similar.
Add to that the garage door opener, the gps and/or all your paperwork in the glove box), and the mistakes start to add up.
Its bad enough they might get your car, but why give them the address to your house, and a means to get in? Its not like youre going to be there when they get there.