← Back to all news
arXiv Machine Learning By Yangze Liu, Zhongyi Han

The Oligarch Barely Steers Model Collapse in Multi-Model Ecosystems

Read the original on arXiv Machine Learning →

The paper investigates whether an oligopolistic concentration of generative AI models accelerates or steers the phenomenon of model collapse when models are recursively trained on each other’s outputs. Using controlled ecosystems of 13 open‑source models and an injected probe that pushes one model’s market share to 90%, the authors find that varying market concentration has little effect on the speed or final state of collapse. Instead, the pace of collapse is largely determined by which models supply the training pool and how susceptible those models are to being carried along, with human‑written text in the pool roughly halving the drift.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 11

A Fragility Spectrum for Recursive Language-Model Training

The paper investigates how recursive contamination—retraining language models on their own generated text—affects output diversity across 13 publicly released checkpoints. Using a fixed contamination protocol over five generations, the authors find a wide spread in 4‑gram diversity (0.187 to 0.940), indicating that some models collapse into repetitive fragments while others remain largely unaffected. The study shows that a model’s susceptibility to collapse is an intrinsic property of the checkpoint, not predicted by parameter scale or static indicators, and that simple interventions such as tightening top‑p sampling can significantly slow or halt collapse.

By Yangze Liu, Zhongyi Han
More like this →
arXiv AI
Sep 1

Moving the Mean Toward the Known Good, Not Beyond It: What Inference-Time Interventions and Weight Consolidation Buy in Open-Ended Generation

The study investigates how inference‑time interventions and weight consolidation affect open‑ended generation in an online bin‑packing task. By iteratively generating, verifying, selecting, and consolidating with LoRA, the model’s outputs shift toward higher value, reducing excess by 1.7 points and outperforming random consolidation by 3.1 points. Across three independent runs, the mean performance remained consistent, and the best candidates converged to the classic heuristic’s level without exceeding it, while consolidation also lowered the proportion of better‑than‑classic candidates but increased their absolute number.

By Roberto I. Ono Filho
fine-tuning
More like this →
arXiv AI
Jun 11

Search Discipline for Long-Horizon Research Agents

arXiv:2606. 11522v1 Announce Type: new Abstract: Autoresearch agents now propose, evaluate, and select scientific candidates against a metric, and that metric is usually an aggregate reduced over a heterogeneous space of regions, slices, or cohorts.

By Adithya Srinivasan, Devesh Paragiri
agents
More like this →