the smaller the rifle, less the muzzle energy, the closer the range required and the more accurate the shot needed to bring down an animal. Not to mention how far a wounded animal is likely to run before expiring, adding to your hunting efforts.
On the farm, we killed pigs with a 22 short--but at point blank range. I would recommend at least a .308 for pigs--tough animals.
For longer range big anaimal, at least a 30-06. I know you can kill them with less but anything worth shooting is worth shooting well.
I have a .338 lapua magnum which is good for 1000 yd shots at elk....not sure when that's gonna happen.