In my opinion, the only downside to building an AR today is just finding the parts in-stock that you need!
I bought both upper and lower receivers stripped from one company (Spikes Tactical), but the LPK from another (Palmetto).
I've mixed Spikes and PSA parts before on two occasions with no problems whatsoever.
Neither Spikes or PSA actually manufacture their stripped lowers in-house. Both get their lowers from companies like Aero Precision and LW Schnieder. Many PSA lower parts come from the same suppliers that supply FN, and supposedly Spike's parts do as well.