Just back in from running some measurements.
Big 'ouch!' here… 
I pulled out the gib and measured it for wear. As expected, there is some difference in thickness along its length. The front end measures 0.1270" (both top and bottom), while the both the middle and back end measure 0.1255" on the top and 0.1260" on the bottom. So a maximum difference of 0.0015" which I suppose isn't too terrible considering the age of this machine.
Unfortunately, there is definite crabbing going on. A dial indicator held out against a ground and polished rod in the three-jaw chuck at about the distance of the end of a boring from the center of the carriage shows a total movement of almost 0.020" between forward vs backward motion (it was even worse before I removed, cleaned, measured and reinstalled the gib). That seems to be way, way more than the wear on the gib could account for and indicates wear at the back of the carriage dovetail.
In the short term, yeah, make boring cuts in one direction only, dial the tool away from the cut when withdrawing. Fussy, but should offer better results than what I'm currently getting.
Longer term fix would be to try to scrape the carriage and/or bed a little to try to get more accuracy into this thing. Of course, I have zero experience with scraping and the rest of the machine is also pretty worn (spindle, bed, etc.) so I do not know if it is worth my time… except as a learning experience. Honestly, I'm beginning to think I might have to rub some cash on this problem – meaning a new lathe – but if I go that route I could always hold onto this and try the scraping when I have nothing to lose.