arXiv AI

Self-Play Pretraining with Zero Data

The paper introduces Self‑Play Pretraining with Zero Data, a proof‑of‑concept method that lets a model generate its own training data by searching over all computable processes using a universal Turing machine. Two models— a generator that proposes byte‑sequence programs and a learner that predicts those sequences—train together, with the generator rewarded for producing data at the learner’s frontier, creating an adaptive curriculum. Experiments show that zero‑shot performance on natural datasets scales predictably with compute, and the models exhibit in‑context learning and discover mathematical sequences during training.

arXiv AI

Sep 10

Generating Pretraining Tokens from Organic Data for Data-Bound Scaling

The paper introduces SynPro, a synthetic data generation framework that augments limited organic text for large language model pretraining by applying rephrasing and reformatting operations. SynPro’s generators are optimized with reinforcement learning rewards for quality, faithfulness, and data influence, and are updated continuously as training plateaus. Experiments on 400M, 1.1B, and 2B models show that SynPro can unlock 3.4–5.2× the effective tokens of standard repetition, even outperforming a non‑data‑bound oracle at larger scales.

By Zichun Yu, Chenyan Xiong

llms reinforcement-learning

arXiv Machine Learning

Aug 12

Scaling Self-Play with Self-Guidance

arXiv:2604. 20209v2 Announce Type: replace Abstract: LLM self-play algorithms are notable in that, in principle, nothing bounds their learning: a Conjecturer model creates problems for a Solver, and both improve together.

By Luke Bailey, Kaiyue Wen, Kefan Dong, Tatsunori Hashimoto, Tengyu Ma

llms reinforcement-learning

arXiv AI

Jul 1

The Label Imitation Game: Turing Test Network for Zero-Shot Pseudo-Label Pruning

arXiv:2606. 30875v1 Announce Type: cross Abstract: Foundation model pseudo-labeling - labeling data strictly via zero-shot inference - enables massive scale, but performance is undermined by hallucinations that evade standard thresholds.

By Brent A. Griffin, Jason J. Corso

llms computer-vision fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Sep 23

Memory-Free Continual Learning with Null Space Adaptation for Zero-Shot Vision-Language Models

arXiv:2510.21175v2 Announce Type: replace Abstract: Pre-trained vision-language models (VLMs), such as CLIP, have demonstrated remarkable zero-shot generalization, enabling deployment in a wide range...

By Yujin Jo, Taesup Kim

llms efficiency multimodal benchmarks

arXiv Machine Learning

Aug 4

Question Begets Question: Self-Evolving Curriculum for Reinforcement Fine-Tuning on Competition Mathematics

arXiv:2608. 01522v1 Announce Type: new Abstract: Teaching a language model a skill it has not mastered is obstructed by three recurring difficulties: training data is scarce, ground-truth reasoning traces are usually unavailable, and models often exhibit an apparent ceiling beyond which additional data yields no further improvement.

By Longtian Bao, Jianyou Wang, Yang Zhang, Youze Zheng, Ramamohan Paturi

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 14

FineInstructions: Scaling Synthetic Instructions to Pre-Training Scale

arXiv:2601. 22146v2 Announce Type: replace-cross Abstract: Due to limited supervised training data, large language models (LLMs) are typically pre-trained via a self-supervised "predict the next word" objective on a vast amount of unstructured text data.

By Ajay Patel, Colin Raffel, Chris Callison-Burch

llms benchmarks

Hugging Face Trending Papers

Aug 6

ProDVI: Programmatic Dynamics Priors for Value Network Initialization

Deep Reinforcement Learning (RL) is notoriously sample inefficient. One contributing factor is that RL agents are typically initialized from scratch, forcing them to acquire task-relevant knowledge through online interaction.

llms agents reinforcement-learning

arXiv AI

6d ago

Self-Play Search Distillation for Large Language Model Reasoning

Self-Play Search Distillation (SPSD) is a framework that generates superhuman synthetic data by having MuZero-like networks play board games in executable environments. The search records are converted into structured reasoning chains that serve as environment‑grounded supervision for training large language models. When applied to Qwen3‑4B‑Base, SPSD improves performance on six mathematics benchmarks from 24.1 to 36.6 and raises the win rate on unseen games from 15% to 45%.

By Lorenzo Molfetta, Wai-Chung Kwan, Giacomo Frisoni, Luca Ragazzi, Gianluca Moro, Pavlos Vougiouklis, Jeff Z. Pan, Pasquale Minervini

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

4d ago

It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs

arXiv:2609.37891v1 Announce Type: cross Abstract: Current pre-training datasets are derived from web crawls, with all their issues, and were not designed to support mid- and post-training pipelines--...

By Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois, Pavel Chizhov, Carlos Rosas-Hinostroza, Neil Si Smail, Benjamin Burtin, Hanna Shcharbakova, Ivan Yamshchikov, Anastasia Stasenko

llms benchmarks

arXiv AI

Jun 6

Image Generators are Generalist Vision Learners

arXiv:2604. 20329v3 Announce Type: replace-cross Abstract: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining.

By Valentin Gabeur, Shangbang Long, Songyou Peng, Paul Voigtlaender, Shuyang Sun, Yanan Bao, Karen Truong, Zhicheng Wang, Wenlei Zhou, Jonathan T. Barron, Kyle Genova, Nithish Kannen, Sherry Ben, Yandong Li, Mandy Guo, Suhas Yogin, Yiming Gu, Huizhong Chen, Oliver Wang, Saining Xie, Howard Zhou, Kaiming He, Thomas Funkhouser, Jean-Baptiste Alayrac, Radu Soricut

llms diffusion computer-vision benchmarks

arXiv Machine Learning

Jul 27

Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning

arXiv:2607. 21971v1 Announce Type: new Abstract: Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains.

By Shujin Wu, Cheng Qian, Xiusi Chen, Heng Ji

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jul 10

Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

arXiv:2607. 07748v1 Announce Type: new Abstract: Large Language Models achieve strong code generation for high resource languages like Python and Java but suffer sharp performance drops on Low-Resource Programming Languages~(LRPLs) such as Julia.

By Didula Samaraweera, Anjana Supun, Srinath Perera

llms reinforcement-learning fine-tuning benchmarks

Related stories