Inadequate bullet-to-barrel fit can result in decreased accuracy. Have you slugged your barrel to determine the groove diameter?
Typically, for a .451" barrel, .452" bullet will provide a good fit.
If your barrel is oversized (.452"+), you may need to use bullets sized larger to improve your accuracy and decrease your leading.
For my .451" M&P45, even lighter target loads with .452" bullets produce accurate shot groups and no leading.