Proper Sight alignment and trigger control is critical with iron sights. While I'm not going into every detail about how to properly align sights and manipulate the trigger without moving the gun, there is one other thing I did not see on the topic when skimming through the responses. Front sight Focus.
When you are shooting iron sights it is critical that your focus is on the front sight rather than on the target. Get your position squared away and solid, keep a natural point of aim so you are not muscularly forcing your sights onto the target, align the sights with the target, control your breathing, FOCUS on the front sight and gently but deliberately manipulate the trigger.
Target focus will produce erratic groups that will drive you nuts and make you question every thing. Practice will reinforce it and shrink the groups. I have seen it many times with experienced competitive shooters who know how to shoot and are doing everything else correct.