It seems impossible to me to change from up to down milling (or vice versa) as the workpiece is rotated around the spindle. So, I don't understand the posting from Den24171. Is he right?
I would suggest making a forked or slotted holder for the spindle, and holding that in the vice. The spindle could be a loose pin or bolt, and there would be much reduced tendency to bend the attachment (and so, less tendency for the cutter to dig in as the spindle bent under load, and to wind the workpiece up the teeth).
Cheers, Tim