[Unclenick]

There's no reason delays can't be made shorter than the eye can perceive, which is about 15 milliseconds. Displays are almost four times faster than that now. Image processing speed is just a matter of how much you chose to do, but will ultimately become no barrier as the low-power microprocessors keep getting faster. I'm sure the military gear doesn't accept any significant level of it.

As to ruggedness, the optics should be the limiting factor, same as in a conventional scope. Solid state electronics can withstand a lot more g-force punishment than a scope sees, and much wider temperature ranges than humans are comfortable in. Electronics components are commonly made available in versions that are specified for -55°C (-67°F) to 125°C (257°F) operating temperatures. You just have to let the designer know you need that and can pay ten times the cost of the least expensive commercial version, which is typically for 0°C(32°F) to 70°C(158°F) operation. There is usually also an automotive version that is not expensive and works form -40°C(-40°F) to 85°C(185°F), though their specs aren't usually looser than standard.

Recoil starts disturbing sight alignment before the bullet arrives on target. How much you can see through recoil depends on how much recoil you have?