Lots of us started like you describe.
What if you don't size your bullets? I size very few bullets. As a matter of fact, the only ones I do size are the ones I want to gas-check.
A small fire, a good pot, a ladle and a mold are all you really need. I'd bet that over the centuries, millions of bullets were cast using those very tools.