Look around for a used machinery dealer, auctions, ebay, craigslist and get a mill. If you find you must use a drill press than get a end mill adapter and dont use the drill chuck.
http://www.shars.com/product_categor..._Mill_Adapters
I had a few endmills laying around that had a MT ground right into them. I have since scrapped them because everything here uses a collet (R-8, Cat40, etc..).
Also run the drill press with the quill retracted and locked.