The problem you will have with using a single powder is that it will not maximize the abilities of all three cartridges. Varget, for instance, would work, but your max loads will leave you 200 fps slower than you could be in the 204. That's why you'll not find many people using it in the 204.
You can certainly do as you please, but having more than one powder is not very expensive (order from Powder Valley, BTW) and you will find that you are able to wring the best performance from all your cartridges.
If I were really bent on those 2 choices, I guess I'd go with the IMR4895, since it appears at first glance to come closest to maximizing all three cartridges.
However, you may well find that one of your guns doesn't like that powder and you'll be off looking for something else anyway.
