Actually, if you lurk around on any of the metalworking usenet groups you'll occasionally hear of someone building a "Gingery Lathe" or "Gingery Machine Shop".
People do it all right, but it's more for the experience than the tools. It's it's own hobby. If you've got a project you need a machine tool for - by all means buy a commercially produced device