Sounds like we better go .50BMG just to be safe.
I don't worry about hunting southern whitetails with a .243. For elk I'd want to go just a little bit bigger. Friend of mine did all his elk hunting with a .270. I reckon that's big enough. I probably wouldn't feel bad with a 6.5 Swede either. If you shoot your rifle enough to know its limits and yours then I think you're good to go.