The big drawback with using a lathe chuck on a rotary-table is the significant loss of headroom.
For screw-on chucks transferred between lathe and mill, the usual way is a mounting-plate with a register on the underside for the centre of the RT, and a boss threaded as with the lathe on the top. The plate has holes to take Tee-nut screws or studs, for clamping to the rotary-table.
NB: arrange some form of locking for the chuck, as milling the work’s periphery will try to unscrew such a chuck from the mounting.
…….
It may be worth making the mounting-plate compatible with not only the rotary-table but also the milling-machine table, allowing direct mounting there.
For a chuck used solely on the mill, directly and on the RT, machine and fit the mounting-plate to replace the usual back-plate.