The only time I've been 'busted' by a deer is when I don't see them first and then move around to spot them after I hear them in the underbrush. Then they suddenly go galloping off.
Deer are very sensitive to movement (like most animals). They are blind to blaze orange or the color of the gun.
I have rounds yet to fire before I sleep ...