I made a rotary broach many years ago, it has a 3mt tanged taper. Holds it well. Don't forget that that the cutting tool will need will be pulled out of the work piece, especially so if the tool is not as sharp as it should be!
The largest hole I've broached is 10mm square in brass with a hardened silver steel tool, that took a lot of pressure! I've also done a 6mm hexagon in mild steel with a HSS tool, again considerable force need but an entirely satisfactory job resulted.
If I were to make the broaching head again I would make it so it would accept commercially available cutters, google "Somma". Heated treated silver steel is fine for cutting brass and aluminium, its a pain making tools from HSS.
The only downside I can see with rotary broaching is that the tools are unserviceable, that is to say you might get one resharpen then its grind the tool down to the next smaller size.
John