Hi Geoff,
My wheels are half your size (I model in Gauge 3) but I don't worry too much about tolerances to be honest. I just try to get them as near the given dimension (in my case G3S standards) as possible. What I do 'do' however is always use a mandrel turned in-situ thus ensuring that all my wheels can be turned to the same diameter (as far as possible) by locking the various slides not in use and putting the same cut on each wheel. For my larger wheels I use a small faceplate with a replaceable nose/mandrel piece that is effectively a consumable and can be turned and die cut to ensure everything is running true. This is of course left in place until all work is completed.
So my advice (if you really want to make adjustments) would be to take a similar approach in terms of mounting your wheels (which you may already be doing of course), fit your "smallest" wheel, touch your tool to this and take the lightest of cuts, lock up and then take the same cut over all the rest of your wheels.
In my view it's much more important to get your wheels the same sizes than to worry too much about tolerances. Also, yours are only 8 thou out (smallest to largest?) and in your scale I'm not sure that's really a problem. But then this is simply my take and others here may (probably will) disagree.
Regards,
IanT