arXiv Machine Learning June 17, 2026 By Hengzhi He, Shirong Xu, Guang Cheng

Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework

Read the original on arXiv Machine Learning →

arXiv:2502. 18049v5 Announce Type: replace-cross Abstract: Recent studies identified an intriguing phenomenon in recursive generative model training known as model collapse, where models trained on data generated by previous models exhibit severe performance degradation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Sep 17

Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data

The paper investigates how to avoid model collapse when training large language models with synthetic data. It establishes theoretical guarantees for the minimum ratio of human to synthetic data needed to maintain training stability, using the Fisher‑Rao metric to analyze dynamics on the probability simplex. The authors derive contraction and invariance bounds that remain meaningful even in high dimensions, showing that the required data ratio differs from earlier estimates.

By Matteo Marchi, Jo\~ao Pedro Silvestre, Bahman Gharesifard, Paulo Tabuada

llms

arXiv Machine Learning

Jul 17

Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence

arXiv:2510. 16657v3 Announce Type: replace-cross Abstract: Synthetic data has been increasingly used to train frontier generative models.

By Bingji Yi, Qiyuan Liu, Yuwei Cheng, Haifeng Xu

arXiv Statistics ML

Sep 10

Learning with Synthetic Data via SGD in High-Dimensional Linear Regression

arXiv:2609.09572v1 Announce Type: new Abstract: Synthetic data has become a promising way to scale model training beyond limited human-generated data but it may also induce strong model collapse (Doh...

By Jichu li, Difan Zou

safety

arXiv AI

2d ago

Rethinking Data Augmentation under Covariate Shift: Invariant-Guided Diffusion and Prototype Reweighting

arXiv:2610.00873v1 Announce Type: cross Abstract: In many industrial applications, 1) tabular data is scarce and imbalanced and thus requires synthetic expansion; 2) input distributions drift between...

By Hongyu Cao, Xinyuan Wang, Arun Vignesh Malarkkan, Kunpeng Liu, Haifeng Chen, Yanjie Fu

diffusion data-engineering

arXiv Machine Learning

Jun 15

Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization

arXiv:2606. 13796v1 Announce Type: cross Abstract: Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution.

By Na\"il B. Khelifa, Richard E. Turner, Ramji Venkataramanan

diffusion

arXiv AI

Jun 16

Quantifying the Impact of Lossy Compression on Neural Generative Surrogate Modeling

arXiv:2606. 15959v1 Announce Type: cross Abstract: Neural networks are used as generative surrogate models for scientific discovery, which are trainable approximations of scientific simulations.

By Zhimin Li, Harshitha Menon, Charles Jekel, Valerio Pascucci, Peter Lindstrom

One email a morning, machine-written

Related stories

Preventing Model Collapse: A Fisher-Rao Perspective on the Dynamics of Training with Synthetic Data

Escaping Model Collapse via Synthetic Data Verification: Near-term Improvements and Long-term Convergence

Learning with Synthetic Data via SGD in High-Dimensional Linear Regression

Rethinking Data Augmentation under Covariate Shift: Invariant-Guided Diffusion and Prototype Reweighting

Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization

Quantifying the Impact of Lossy Compression on Neural Generative Surrogate Modeling