Speed isn't critical, but you'll have to reduce feedrate accordingly. There are three main issues. One, how to hold a small drill. I use an Eclipse pin chuck in the main drill chuck. Second is how to start the drill. At these small sizes a centre drill, or even a spot drill, may leave too large a spot, without a clean centre. I usually just let the drill gently touch the work and it'll find it's own centre, especially if it's 4 facet. Third, the main killer is chips getting jammed, so peck very frequently and use a thin liquid to provide some lubrication and to wash the chips away. I use WD40, mainly because it's handy in a spray bottle. The smallest holes I've drilled were 0.7mm and 10mm deep, albeit in brass. But the drilling was done on a Bridgeport, which doesn't make it easy.
Andrew