My two cents
If you cast you own bullets and use metal in which he lead content is down to 95% (and consequently the hardness is a bit high) I would choose the smallest ball that shaves a good ring. This is especially true if you are not using a press. I have used .451s in Pietta ASM, ASP, and Colts but they all seem to like .454s. Actually the pistols don't care. It is I who likes the larger bullets. I cast my own and the bullets are a tiny bit hard.
If you are buying bullets with a high lead content, you can get away with a larger ball.
My reading of history convinces me that most bad government results from too much government. Thomas Jefferson