Make sure to clean your die also. An accumulation of lube and gunk can mess you up. I clean mine each time with an aerosol gun cleaner then clean it out with a patch until the patch comes out clean, just like cleaning your rifle. Also use a flashlight and visually inspect the die inside.
I also just use a touch of lube on my thumb and forefinger to lube cases. I don't use the lube pad technique anymore.