As Dave Halford says, we need more information.
What machine are you using?
We assume that the workpiece is being held securely, and unable to shift.
Both work and cutting tool must be held securely
What is the material? HSS will have a job cutting hardened steel. You need carbide for that.
How does the lathe behave with different material? It could be irregularities in the material.
Is the tool sharp?
What sort of tool is is? A form tool, having a greater cutting edge might be more liable to chatter that a single point knife tool.
Is it at centre height?
Is it set to present just the cutting point to the work, rather than a long edge?
Are the gibs tight enough to prevent movement?
Whar depth of cut are you using?
What feed rate are you using? If either of these is excessive for the tool/material combination, you may well have problems.
What cutting speed are you using? Too high a peripheral speed will shorten tool life, and cause problems.
Finding the answers to some of these questions may help you to find the solution, rather Forum members without the detailed knowledge that only you can have, being on site.
Howard