Well I just started loading about 2 weeks ago so maybe I have some input.
I read tons of stuff (books, forum postings, ect). In the end I bought the Lee 50th anniversary kit. So for I have been %120 happy with it. It included everything (except dies) to start loading and only cost $130. Pretty cheap investment to start with.
My decision was mostly influenced by the fact that I only shoot 9mm right now. It was pretty hard to justify spending $400 + on a press to save $8 a box.
I have now loaded about 1500 rounds with it and shot 500 of those. So far the rounds have been perfect. When I first got it, it took me 5 hours to load 400 rounds. Now I can load 120 - 130/hour. A lot faster than I was lead to believe on a single stage.
I still believe a single stage press is the right place to start. My plan down the road, is to continue to load 9mm with and soon begin loading 45acp. Once I have loaded enough ammo to save the cost of the press times 2 I will sell it and buy a progressive. By then, I should be pretty proficient at loading.