The extra weight should help to compensate a little bit for the muzzle rise, and leaving it off makes it more sleek and handy.
Choosing whether to keep it on or not depends on how you feel when you shoot it.
Shoot it both ways and then whichever configuration it's left as will end up proving to be the most suitable.