When I got started, I bought a Lee Aniversary set for about $75, Plus a shelholder. That cost me a couple of bucks more.
$15-20 for powder, $2-3 for primers...range brass and bullets ($ depending on what you are reloading).
Off and running.
I then welded a plate that I screwed my press onto. Recessed the heads of the screws so it sat flush on the table.
For a few years, when I had some empties that needed to be restuffed, I laid some newspaper on the dining room table and C-clamped the plate to the table.
I never scratched the top and the wife never complained.
IOW yes, you can pack it in the closet and load when the need arises...or the urge becomes compelling.