Neat milling-spindle design.
How did you hold your milling samples?
In the tool-post, as the illustrations of the milling-spindle imply?
If so I suggest making a milling-table that fits the cross-slide in place of the tool-holder, with either Tee-slots or a matrix of tapped holes, and provided with suitable clamps.
It would be much more rigid, and give more headroom under the cutter – the corollary being deeper engagement of the spndle body and vertical-slide for lower-profile work, making that area more rigid.
(The obvious disadvantage of using holes, offset to some extent by arranging some sort of covers, is they and their threads trapping swarf.)
To secure the milling-table to the cross-slide, if I am right that the central channel is a broad Tee-slot, I’d use at least three generously-length T-nuts whose upstands fit the channel snugly, and take socket-head screws counterbored into the table. Ensure the screws cannot pass through the T-nuts and act on the base of the T-slot, as that would be a recipe for disaster by breaking the slot’s flanges.