arXiv AI

Break Step: Recursive Training Resonates with Replayed Sampling Noise

arXiv Machine Learning
Sep 11

A Fragility Spectrum for Recursive Language-Model Training

The paper investigates how recursive contamination—retraining language models on their own generated text—affects output diversity across 13 publicly released checkpoints. Using a fixed contamination protocol over five generations, the authors find a wide spread in 4‑gram diversity (0.187 to 0.940), indicating that some models collapse into repetitive fragments while others remain largely unaffected. The study shows that a model’s susceptibility to collapse is an intrinsic property of the checkpoint, not predicted by parameter scale or static indicators, and that simple interventions such as tightening top‑p sampling can significantly slow or halt collapse.

By Yangze Liu, Zhongyi Han
arXiv Computation and Language
Sep 25

What a Cross-Model Fixed-Point Census Can and Cannot Arbitrate About Repetition

The paper investigates neural text degeneration by measuring the fixed‑point structure of short‑window argmax maps across 17 pretrained models, using 96 random two‑token starts without prompts. It finds a stable four‑way classification that varies across model families and scales, with some models funneling to a single endpoint token while others do not, and shows that this behavior is not solely determined by training data or corpus frequency. The study demonstrates that repetition phenomena are not uniformly explained by either training data or network architecture alone, highlighting the complexity of neural text generation dynamics.

By Nicol\'as Vera Z\'u\~niga
arXiv AI
Jul 3

Procedural Memory Distillation: Online Reflection for Self-Improving Language Models

arXiv:2607. 01480v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR), along with recent selfdistillation variants such as SDPO, evaluates each rollout against a verifier and updates the policy from that episode-level signal.

By Ye Liu, Srijan Bansal, Bo Pang, Yang Li, Zeyu Leo Liu, Yifei Ming, Zixuan Ke, Shafiq Joty, Semih Yavuz
arXiv Computation and Language
Oct 27

Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation

arXiv:2509.15194v4 Announce Type: replace-cross Abstract: Large language models (LLMs) can improve through reinforcement learning with verifiable rewards (RLVR), but many settings lack reference answ...

By Yujun Zhou, Zhenwen Liang, Haolin Liu, Wenhao Yu, Kishan Panaganti, Linfeng Song, Dian Yu, Xiangliang Zhang, Haitao Mi, Dong Yu
arXiv Computation and Language
Oct 22

ECHO: Emergent Common-Mode Hidden Offsets Drive Vocabulary Collapse in Preference Optimization

The paper investigates how preference optimization can inadvertently create a shared directional shift—termed a common‑mode offset—in a model’s hidden states. This offset, which affects both chosen and rejected responses, accumulates without being penalized by the ranking loss, eventually projecting onto the output layer to form a vocabulary well that dominates decoding and leads to generation collapse. Experiments across multiple base‑model lineages and fine‑tuning regimes confirm that masking this offset or using per‑token ranking losses can prevent the collapse, while standard ranking metrics fail to detect the risk.

By Yun Li, Ehsan Javanmardi, Simon Thompson, Jiachen Gong, Qunli Zhang, Zifan Zeng, Shiming Liu, Peng Wang, Zixuan Guo, Manabu Tsukada
arXiv Computation and Language
Oct 28

3, 7, 14, 17, 42: Instruction Tuning Installs and Sharpens Number-Choice Heuristics in Language Models

The study measures the exact probability that language models assign to each integer from 1 to N when prompted for a random number, eliminating sampling noise by summing over all spellings. Across 18 open‑weight models and 29 decoding settings, instruction tuning consistently makes the distribution less uniform, introducing a round‑number avoidance heuristic and sharpening endpoint avoidance. While tuned models rarely output the extreme values 1 or N and largely avoid multiples of 10, they do not converge on a single favorite integer across models.

By Samar Ansari
arXiv Computation and Language
Oct 16

Context Injection and Long-Horizon Reliability: Separating Duplicate Recognition, Local Processing, and Counter Tracking

The paper introduces a 400‑step ledger task that evaluates language models on three granular components: duplicate recognition, local processing, and counter tracking. Experiments on Qwen3‑8B show that duplicate‑recognition errors increase only late in the episode, while the counter remains flawless over 8,000 steps and local processing stays stable. Adding irrelevant content before each step causes the counter to fail consistently, but an external reminder can restore its accuracy.

By Thomas Serval
arXiv AI
Sep 18

Chronicle: Cut-Point Replay for Regression Testing of LLM Agents

Chronicle introduces a method called cut‑point replay to make regression testing of large language model (LLM) agents reproducible. It records an agent’s run at non‑deterministic boundaries as immutable envelopes and then replays selected boundaries while executing the rest live, enabling continuous‑integration tests that detect faulty code changes. Benchmarks show minimal overhead, perfect bit‑stability, and effective detection of unsafe actions in a mutation study.

By Tisha Chawla, Susheem Koul