Get on Craig's list and you can find used single stage presses for cheap. The old RCBS junior series are very good presses and can be had for 30 bucks. Those presses are heavy duty and are strong enough to load rifle cartridges. Using a press is so much easier and faster than the little kit. Save yourself some agony and just get a press. The costs of everything else are still the same. If you use a lee auto disk powder measure on a single stage it speeds things up considerably. Just something to keep in mind as you get more money.
Besides dies the next necessity is a scale. You can't load without one, period. Be sure to read and reread a reloading manual. Read here as well. There's a bit to learn but reloading isn't too difficult especially if you are mechanically inclined. You can always depend on getting help here.
You will enjoy reloading besides the fact it saves quite a bit of money.