Yes, you don't want to sand down the collet that will INCREASE the diameter. You have to determine if the problem is the mandrel or the collet. The case will have the neck squeezed by the collet, then the mandrel is pulled back up through the neck to push it out to around minus 0.003" - 0.006".
I tried to simplify it in my post above but this is a more detailed description of the process of neck sizing.
So, if the collet is not small enough then it may not reduce the neck enough for the mandrel to even have contact. You can test this by removing the mandrel and see how the bullet fits in the neck after sizing with just the collet.
If the collet is reducing it properly then the mandrel may be too big. You can either ask for a replacement or sand it down.
My RCBS .223 Rem die has a mandrel diameter of .222" and a sized case has an inside neck diameter of .218"-.219". Bullets of course are .224" so trying to stuff a .224" bullet into a .218" neck is not happening by hand.
"The ultimate authority ... resides in the people alone. ... The advantage of being armed, which the Americans possess over the people of almost every other nation ... forms a barrier against the enterprises of ambition."
- James Madison