One suggestion is to do a search of this site (and others) on "gunsmith"; a tremendous amount has already been posted on the subject, mainly about the need to be a businessman, not a hobbyist tinkerer. And the need to learn gunsmithing. Too many folks think all they need is a mill bastard file, a screwdriver and an FFL. We have too many of that kind, learning on customers' guns.
You sound like you are taking a sensible approach and of course your retirement income will allow you to keep eating while building your business. Except in a very few places, if you do good work and keep good customer relations, you will soon have more work than you can handle.