The two basic families of materials being discussed here are either crystalline or amorphous.
I'd like to suggest (although I haven't tried), that polishing items of crystalline materials, e.g. polyacetal, Nylon, polyethylene, polypropylene, etc. need to be deep frozen, or at least taken below their glass transition temperature.
I have machined (trued up) with some success, polyester (rubber) skate wheels directly from the freezer.
I also question, during the abrasion and polishing, if the surface of crystalline materials is being continually 'pulled apart', thus introducing (let me call it), a broken surface. Someone with good microscopy (and photographic) skills might like to intervene here.
Acrylic, and other amorphous materials on the other hand, are (generally) a breeze to polish.
Good luck,
Sam