The handguard retainer is the metal part at the front.
Any metal at the rear will come off with the wood.
The only metal usually on the rear of the handguard is either a metal cap-like cover, or a flat spring to give the wood tension.
Either will stay attached to the wood.
Wiggle and pull harder, and make sure the retaining lever on the right front of the wood stays pointed forward.
Here's a link to a site with photos of how to remove the wood.
The link is on the lower left of the page: "Furniture Removal".
http://www.gunsnet.net/linx310/