Tried taking some more of to get rid of them but it doesn’t seem to have made any difference
When you say you tried taking more off do you mean you were able to remove material but the depressions were still there or that you were not able to remove any more material?
Looking at those two surfaces it looks like you have just tried to take a very shallow cut as there are still signs of the cast surface. This is not really the ideal approach as the outer skin can be hard even on a non chilled casting which will not cut easily and just knock th eedge off your cutter.
First thing would be to check your cutter is still sharp and take say a 1mm (0.040″) cut off the bottom first*. The box bed has enough height for this. If OK repeat for the top.
If the cutter won’t touch the metal then the next option is to change to a carbide cutter which is more likely to get through the skin and also any chill.
Failing that if the casting is really chilled then you need to get it upto red heat and hold it there for about an hour then allow to slowly cool. This time of the year the BBQ can be put to use for this.
* You don’t really need to machine the bottom, if it is quite uneven then a fit of file work will knock off the high spots. Then lay a sheet of Emery Cloth 120g on a flat surface and rub the casting back and forth, so long as the two ends clean up you will then have a flat surface that can be clamped to the mill table so the top can be machined