arXiv AI
A Dominant Supplier Slows Recursive Drift More Than It Steers It
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
4d ago
Break Step: Recursive Training Resonates with Replayed Sampling Noise
arXiv:2609.11149v3 Announce Type: replace-cross Abstract: How fast does a language model degrade when trained on its own outputs? Theory traces it to gradually accumulating errors, while experiments...
arXiv Computation and Language
Sep 25
What a Cross-Model Fixed-Point Census Can and Cannot Arbitrate About Repetition
The paper investigates neural text degeneration by measuring the fixed‑point structure of short‑window argmax maps across 17 pretrained models, using 96 random two‑token starts without prompts. It finds a stable four‑way classification that varies across model families and scales, with some models funneling to a single endpoint token while others do not, and shows that this behavior is not solely determined by training data or corpus frequency. The study demonstrates that repetition phenomena are not uniformly explained by either training data or network architecture alone, highlighting the complexity of neural text generation dynamics.
arXiv Machine Learning
Sep 11
A Fragility Spectrum for Recursive Language-Model Training
The paper investigates how recursive contamination—retraining language models on their own generated text—affects output diversity across 13 publicly released checkpoints. Using a fixed contamination protocol over five generations, the authors find a wide spread in 4‑gram diversity (0.187 to 0.940), indicating that some models collapse into repetitive fragments while others remain largely unaffected. The study shows that a model’s susceptibility to collapse is an intrinsic property of the checkpoint, not predicted by parameter scale or static indicators, and that simple interventions such as tightening top‑p sampling can significantly slow or halt collapse.
arXiv Machine Learning
Sep 11
The Oligarch Barely Steers Model Collapse in Multi-Model Ecosystems
The paper investigates whether an oligopolistic concentration of generative AI models accelerates or steers the phenomenon of model collapse when models are recursively trained on each other’s outputs. Using controlled ecosystems of 13 open‑source models and an injected probe that pushes one model’s market share to 90%, the authors find that varying market concentration has little effect on the speed or final state of collapse. Instead, the pace of collapse is largely determined by which models supply the training pool and how susceptible those models are to being carried along, with human‑written text in the pool roughly halving the drift.
arXiv AI
Sep 15
LLM Probability Concentration: How Alignment Shrinks the Generative Horizon
arXiv:2506.17871v4 Announce Type: replace-cross Abstract: Despite their impressive capabilities, aligned large language models (LLMs) often generate outputs that lack diversity. What drives this cons...
arXiv Machine Learning
Sep 10
Do Reviewers Still Reward Lexical Complexity? A Frozen-Rater Study of Preference Drift in 124K ICLR Reviews
arXiv:2609.08475v1 Announce Type: cross Abstract: Large language models have collapsed the cost of producing lexically elaborate prose, and whether peer reviewers still reward it is a question about...