I have no experience of using a drill motor in the way you describe, but I would have thought that it wouldn't have sufficient speed for the type of application which you are envisaging. I built a couple of spindles using sewing machine motors. They are clamped in the toolpost and can be used at any angle to the lathe axis.:

These motors are only rated at about 90W – so they are never going to rip great chunks of metal off. They do however, rev to about 10,000 rpm (according to the blurb) so are suitable for small mills and even engraving. For scale, the collet is an ER25. By substituting the "spare" pulleys on the right of the picture, the speed can be increased even further, albeit with a loss of torque, of course. These sewing machine motors come with a three-step variable speed pedal arrangement, which makes for exciting hand/eye/foot coordination!
Hope this helps,
John
Edited By John Hinkley on 04/12/2016 09:59:57