I have made a lot of small game front stuffers for customers over the years, as small as 25 cal and up to 40. 45s are used a lot on small game, but they start to get into the "deer rifle" category.
The 2 most popular calibers have been 40 and 36, but 32 is asked for a lot too.
Most of them are zeroed dead on at 25 yards. It's common to have them shoot 5 balls through a hold as big as a dime at that range, so any small game animal is in real danger from one of these rifles if the person holding it can shoot.
The advantage of the 40 over the other calibers is that it shoots balls that weigh about 98 grains and therefore do not drift as much in the wind as the smaller balls, and also that a 3/8" rod will load a 40 cal, but is too big to enter the bore of a 36 cal or anything smaller. For 32 and 36 cals you can use a 5/16" rod. A 30 or smaller needs a 1/4" rod.