I would suggest compromising between the Lee setup and a progressive loader. Buy a single stage press like the RCBS Rockchucker and a three die set and shellholder from RCBS, Lee, Redding, etc. Get a powder scale and some measuring dippers, or a scale if you want to speed it up a little. Buy several reloading manuals, for example the Speer or Sierra.
Then do some research to find some lead bullets and a suitable powder that shoot well in your rifle and buy them in bulk.
Once you are comfortable, work up some jacketed bullet loads as well. This set up isn't as fast as a progressive loader, but you can safely load some very high quality ammunition a lot faster than with the Lee hand press.