You would be better off drilling and tapping a hole in the shaft and using a screw, rather than use plastic or brass. If this is for fixing a pulley onto a motor, then definitely not plastic.
If you cannot broach a slot in the shaft, then file one! With a little care, it is feasible to get a good fit. Good practice at filing too!
Andrew.