Its the anvil
The anvil has enough spring to tightly grip the decapping pin. Polishing it to have a spherical tip will solve this.
They are hard and the only way is with an abrasive stone. Fortunately it doesn't take much, just 5 minutes with it chucked in a drill.