I am really worried about posting on these things because there is a huge risk of teaching Granny how to suck eggs and getting accused of all sorts of Smart Alecry. I've only been on here a few months and I didn't read the whole site, but In for a penny in for a pound, here we go…
I would guess machine and tool shank bending. We all know the difficulty in machining off that last thou. You advance .001" and make the cut, no difference. You repeat, nothing and then blammo, .004" undersize.
The tool puts up resistance to the cut, it would rather rub and bend everything than make the cut. Only when it can no longer resist the cutting pressure does it make the cut and then it takes everything available.
There has to be enough pressure to get your cutting edge under the surface. I have a notion that enough cutting teeth means one is always engaged and hold the tool against that surface when the next one arrives.
A scalloped surface may be indicative of a tool that is oscillating between rubbing and cutting. Everything bends while it builds up the cutting pressure and then blammo. To counter an oscillation you change feeds and speeds hoping to find a natural damping. You reduce overhangs wherever possible. You cut down hill to try and minimise the overshoot. If this were a production environment you might even consider unequal helix tooling.