I'm not sure why, but the empty magazine makes the slide and slide release harder.
That is by design.
When the magazine is empty, the magazine follower bears upward against the slide release. That is the mechanism by which the gun locks the slide open after the last round is fired. With any rounds in the magazine, the follower is too far below the slide release to contact it and therefore it applies no upward pressure on the slide release.