4-flute cutters are not good for cutting slots, try a 2-flute one also known as a "SLOT Drill" for now obvious reasons.
Basically as the lead tooth on a 4 flute takes a cut it will pull the cutter sideways and the following tooth will take material off the side of the slot. With a two flute cutter while one flute is cutting the other is not in contact with metal so won't make the slot wider.
Slop in teh mill could also cause problems.
So if you want an accurate slot use a good sharp 2-flute cutter at full width or a smaller one and take 3 passes, one to remove most of the material down the middle and then a sizing cut down each side