My wife is 5'1" and 120lbs - she has had no troubles with my XDMs or Glocks. The only thing she doesn't like is the 1911, because the plunger in the front is a bit tough.
It sounds like you just need to learn how to use leverage in your favor.
Hold the gun in your right hand in a shooting style grip. Point the gun to the left and get a firm grasp on the top of the slide. Force your hands together to move the slide to the rear, rather than sling shot style - my wife has trouble with sling shot style, too. With this method, you're using your upper arms and shoulders to compress the spring.
Kind of difficult to explain, I suppose, but look up YouTube videos. I'm sure there are some demonstrations.
Glock 17 G4 FDE | S&W M&P 22 | Sig P226 MK25 | Sig P229 Extreme | HK P30 | HK45 | HK USP | HK USP Compact | HK USP 45 TACTICAL | XDM 45 Compact | XDM 9
Daniel Defense DDM4 V4 | Arsenal SGL 21 | ASA SBR w/ Side Charger | Savage Mark II FV-SR | PSA M4 clone
Mossberg 590A1 20" SP | Mossberg 500 Cruiser