Graphited yarn has been used for years with a great deal of success. You need a suitable cavity in which to fit the sealing medium, which is, as you surmise, compressed by tightening up the gland.
Depending on how the crosshead is attached to the piston rod, it may be possible to use O rings. Some have used conical-shaped PTFE seals whereby tightening the gland compresses the PTFE against the piston rod.
Whatever system of seals you use, the piston rod should be a very smooth finish, or you'll wear your seals out very quickly.
Have a look at some Stuart Turner or similar drawings; Model Engineer has shown the arrangement of seals for locos (eg Martin Evans, LBSC, etc), traction engines and stationary engines (eg Anthony Mount) over the years.