I just started myself. I bought a Lee kit and dies and am very happy with it. (I had to keep my costs WAY below $500) Currently I am loading 9mm and just started working on some .270 loads. I will eventually be loading for at least a half dozen more calibers, and will probably at some point buy a progressive but I decided that to get started a single stage was safer and easier to learn on, and I will load all of my hunting ammo individually. If you get a steady pace established you can cycle through a bunch of brass pretty quickly, though not nearly as fast as a progressive.
Sensop was right on in his advise to buy at least one manual or book and read it before you buy any other equipment. I already have the Lee, Nosler, and Hornady manuals, as well as the small freebie load data pamphlets from several companies.
A buddy of mine mostly uses Iosso (sp.?) case cleaner rather than his tumbler, you just soak the cases in it and swish them around then rinse and dry, gets them clean but not polished. I will probably buy a chrono soon, as I will be really looking for consistency as well as velocity in the hunting loads for my .270, .30-06, and a couple of other rifles.