When EOS Tokens Disagree: Understanding Length Inflation in On-Policy Distillation

The paper investigates why on‑policy distillation (OPD) can produce excessively long student responses, attributing this to a termination‑token mismatch between base students and post‑trained teachers. Experiments on Qwen3, Llama, and Gemma show that differing stopping probabilities for identical EOS token sets suppress the student’s preferred termination and fail to transfer the teacher’s alternative. Aligning decoding stopping sets alone is insufficient; treating functionally equivalent EOS tokens as a shared semantic stopping action reduces length inflation, though late‑stage inflation persists beyond termination alignment.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• efficiency
• safety