Fillers are a bad idea - ringed barrels. The only time a filler is required is when loading 2gr of Bullseye. Otherwise the right way to do it is to use a pistol powder if you want a plinking load.
Some rifle powders, like H4895, can be reduces quite a ways safely - but it sure doesn't burn clean (you'll throw powder zombies everywhere).
Here is some info on plinking loads:
Low Velocity Data