The more I think about this problem, the worse it seems. I think the problem of achieving a tiny, known, controlled displacement has many similarities to measuring such a displacement. After all, a micrometer measures by producing displacement.
But there's really difficult, more fundamental problem, which, until solved, prevents further progress. How does one detect the initial contact of probe to surface, as the probe advances, before the electronic detection occurs?
It gets worse: for the probe to produce meaningful results, its ball-end's diameter must be known (presumably to sub-micron accuracy), as well as the amplitude of its vibration at the times when electronic detection occurs, as well as at initial contact. In other words, the envelope produced by the vibrating ball's surface must be known (again to sub-micron accuracy). Also, to be useful in practice (to sub-micron accuracy), the centre of the ball must be aligned with the machine spindle to sub-micron accuracy. And then there are the standard metrology bugbears of the effects of temperature, air movement, surface contamination, gravity, etc., etc.. It's a horrible collection of problems!
I still think that optical methods might be able to solve more of the problems than purely mechanical methods, but it sounds like very large magnification would be required, so microscopy might be out (oil-immersion objectives don't sound practicable here). What about projection methods?
I know little about electronics, however, a feedback system is described. Can the feedback sensor's signal be used to, for instance, detect initial contact, before resonant vibration is damped by heavier contact? One might hope that the hard ball, clattering against a stiff-enough surface, at initial contact, might produce an albeit tiny, detectable signal, or at least distort (clip?) the feedback signal's waveform.
I can think of other problems, but it seems bad enough already. Glad it's not my problem… Most impressive – good luck!