Other one extracted perfectly and I cleaned it in its entirety. The one on the right is just stuck. It doesn't look welded in but it could be rusted/glued in. It looks like the guide rod snapped as the set screw is still present.
I'm almost of the feeling I could make a replacement part myself out of some wire and tap and die but I don't have anything near that small. I just can't get the original out.