Hi Alfie
I would suggest you make and use a draw bar, tapping the collet chuck to secure in the taper or to remove from the taper is not good for the headstock bearings. I made mine using threaded rod and a top-hat threaded heavy washer at the spindle end.
If you have a gap at the chuck to spindle point you can make and use a wedge or a pair of wedges to remove the chuck.
Emgee