On a beam balance scale, the rate at which you dump the powder into the pan should NOT make any difference in the final reading. If it does, then you probably have dust on the scales knife edge and balance point. Clean them carefully with a Q-tip and make SURE you don't leave a fine cotton strand from the Q-tip when you do that.
On electronic scales, I also find that I can slowly trickle powder into the pan without changing the reading, then pick the pan up and set it right back down and get a different reading. (That is one reason I have not decided to pay >$200 for an automatic charge throwing device that is based on an electronic scale.)
I am betting that the correct weight is what your scale says when you remove and replace the pan. You can find out by checking with a beam balance. As you noted, the difference between where the scale stops the powder dispenser and the weight at pan replacement is NOT always the same. And I don't have any good ideas on how to make it the same so that you can make an offset and realiably get a charge to within 0.1 or 0.2 grains, or even 0.5 grains.