First, there is a chance that some of the parts you build may not be completely compatible, or not completely in-spec. By that I mean that things like trigger pins or take down pins that are too loose/tight or just not fitting at all without a little modification. For example on my 2nd AR15, I bought both upper and lower receivers stripped from one company (Spikes Tactical), but the LPK from another (Palmetto). The trigger pin seems to be 'walking out' when first shot 40rds to test the rifle.
The FCG is not installed correctly. The hammer pin is secured by a spring (J or D) in the hammer and the trigger pin is secured by the hammer spring legs. Hammer pin has a groove in the center and the trigger pin at each end. Spring tension in these grooves is what keeps the pins from walking.