About the only thing I'd consider using filler with is my .375 H&H cast bullet loads with 2400 or 4227. The case is just huge in comparison to the size of the powder charge.
I can't think of any handgun cartridge that I would consider using filler with. I've got quite a bit of .357 mag ammo loaded up with 158 grain cast SWC and 7 grains of Unique. Definitely no filler needed with those loads.