arXiv AI

Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Models

arXiv:2410. 12341v4 Announce Type: replace-cross Abstract: As AI-generated content increasingly populates the web, generative AI models are at growing risk of being trained on their own outputs, a process known as AI autophagy.

arXiv Machine Learning

Jul 31

Epistemic diversity across language models mitigates knowledge collapse

arXiv:2512. 15011v3 Announce Type: replace Abstract: Artificial intelligence (AI) increasingly generates the very content used to train future AI systems.

By Damian Hodel, Jevin D. West

llms

arXiv AI

Jul 17

Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

arXiv:2510. 01171v4 Announce Type: replace-cross Abstract: Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse.

By Jiayi Zhang, Simon Yu, Derek Chong, Anthony Sicilia, Michael R. Tomz, Christopher D. Manning, Weiyan Shi

llms safety

arXiv AI

Jun 26

Limited Reference, Reliable Generation: A Two-Component Framework for Tabular Data Generation in Low-Data Regimes

arXiv:2509. 09960v2 Announce Type: replace-cross Abstract: Synthetic tabular data generation is increasingly essential in machine learning, supporting downstream applications when real-world, high-quality tabular data is insufficient.

By Mingxuan Jiang, Keyang Chen, Yongxin Wang, Yongsheng Zhao, Ziyue Dai, Yicun Liu, Zeping Li, Qiuyang Zhang, Hongyi Nie, Hongbin Zhu, Sen Liu, Guangnan Ye, Hongfeng Chai

llms diffusion fine-tuning benchmarks safety

arXiv AI

Aug 25

Reviewing Model Collapse and Countermeasures

arXiv:2608.21366v1 Announce Type: new Abstract: Driven by massive amounts of web-scale data, generative AI (GenAI) has achieved remarkable progress, enabling various applications in diverse sectors....

By Xihao Xie, Beichen Hu

arXiv AI

Jul 21

Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning

arXiv:2505. 20161v2 Announce Type: replace-cross Abstract: Effective generalization in language models depends critically on the diversity of their training data.

By Jaehun Jung, Seungju Han, Ximing Lu, Skyler Hallinan, David Acuna, Shrimai Prabhumoye, Mostafa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi

llms benchmarks

arXiv AI

Jul 22

Large Language Models Explore by Latent Distilling

arXiv:2604. 24927v2 Announce Type: replace-cross Abstract: Generating diverse responses is crucial for test-time scaling of large language models (LLMs), yet standard stochastic sampling mostly yields surface-level lexical variation, limiting semantic exploration.

By Yuanhao Zeng, Ao Lu, Lufei Li, Zheng Zhang, Yexin Li, Kan Ren

llms benchmarks

arXiv Machine Learning

Jul 7

Language Generation with Replay: A Learning-Theoretic View of Model Collapse

arXiv:2603. 11784v2 Announce Type: replace Abstract: As scaling laws push the training of frontier large language models (LLMs) toward ever-growing data requirements, training pipelines are approaching a regime where much of the publicly available online text may be consumed.

By Giorgio Racca, Michal Valko, Amartya Sanyal

llms

arXiv AI

Sep 15

Generate to Explore, Select to Exploit: Aligning LLM-based Headline Generation with Personalized Recommendation

arXiv:2609.15094v1 Announce Type: cross Abstract: In industrial recommendation feeds, presenting a static headline for an item often fails to satisfy the diverse, multimodal interests of the user pop...

By Yi Chen, Rufeng Cheng, Qiang Xie, Tao Li

llms multimodal benchmarks

arXiv Machine Learning

4d ago

It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs

arXiv:2609.37891v1 Announce Type: cross Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--...

By Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois, Pavel Chizhov, Carlos Rosas-Hinostroza, Neil Si Smail, Benjamin Burtin, Hanna Shcharbakova, Ivan Yamshchikov, Anastasia Stasenko

llms benchmarks

arXiv Machine Learning

Jun 4

Data Attribution in Large Language Models via Bidirectional Gradient Optimization

arXiv:2606. 04928v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed across diverse applications, raising critical questions for governance, accountability, and data provenance.

By Fr\'ed\'eric Berdoz, Luca A. Lanzend\"orfer, Kaan Bayraktar, Roger Wattenhofer

llms safety

arXiv AI

Jun 16

LLM Jaggedness Unlocks Scientific Creativity

arXiv:2605. 10574v3 Announce Type: replace Abstract: As artificial intelligence advances, models are not improving uniformly.

By Shray Mathur, J. Anibal Boscoboinik, Esther H. R. Tsai, Kevin G. Yager

llms benchmarks

arXiv AI

Jun 17

R1-SyntheticVL: Is Synthetic Data from Generative Models Ready for Multimodal Large Language Model?

arXiv:2602. 03300v2 Announce Type: replace-cross Abstract: In this work, we aim to develop effective data synthesis techniques that autonomously synthesize multimodal training data for enhancing MLLMs in solving complex real-world tasks.

By Jingyi Zhang, Tianyi Lin, Huanjin Yao, Xiang Lan, Shunyu Liu, Jiaxing Huang

llms multimodal benchmarks safety

Related stories