Rest assured, you're not the first one to be found in this situation. As
hk33ka1 noted, you can get a basic do-all kit for less than $100. Have you used a cost calculator to see how quickly you'll break even reloading? Here are links to ones for
rifle and
pistol loads. They may give you a better idea of what your actual costs and savings may be.
Your biggest savings can be realized reloading your classic pistol cartridges with home cast bullets and light charges of hi-energy powder (like Bullseye). The primers are basically the same price as your rifle loads but the powder and bullets may be only about 1/10 as much. With home casts, I load my plinker/target .38-SPL WCs, .357-Mag RNFP, and .45-ACP SWCs for around the same cost as quality .22LRs.