The paper introduces two new sampling techniques—GROOT and Verbalized Sampling—to create strategically diverse training data for self-training large language models. By focusing on substantive variation in problem-solving approaches rather than just correctness, the authors demonstrate that models trained on this data outperform those trained on IID samples across competitive programming and Next‑Chapter Prediction tasks. Notably, self‑training with strategically diverse but incorrect traces from Qwen3‑4B surpasses IID distillation from a 235B teacher, challenging assumptions about the importance of correctness and teacher scale.
arXiv:2606.29985v2 Announce Type: replace Abstract: Diversity in LLM mathematical reasoning is critical for exploration, but common diversity metrics mostly capture surface-level variation rather tha...
Diversity in LLM mathematical reasoning is critical for exploration, but common diversity metrics mostly capture surface-level variation rather than differences in how a problem is solved. We address this gap by introducing approach-level diversity: variation in strategies across correct solutions to the same problem.
arXiv:2502. 11027v5 Announce Type: replace Abstract: Large language model (LLM) scaling inference is key to unlocking greater performance, and leveraging diversity has proven an effective way to enhance it.
arXiv:2510. 11686v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) promises to expand the capabilities of language models, but it is unclear if current RL techniques promote the discovery of novel behaviors, or simply sharpen those already present in the base model.
arXiv:2510. 16882v4 Announce Type: replace-cross Abstract: Supervised fine-tuning (SFT) is a commonly used technique to adapt large language models (LLMs) to downstream tasks.
The paper proposes a new approach to large language model (LLM) reasoning that moves beyond naive repeated sampling. Instead of generating many independent solutions, it first samples problem‑specific concepts, hints, or strategies and conditions answer generation on them, producing a single trajectory of diverse concepts. A small concept generator is then trained via reinforcement learning to maximize downstream success, leading to significant improvements in pass@k on hard mathematical reasoning tasks compared to both naive sampling and concepts from larger untuned models, and the trained generator transfers to unseen answer generators, including those from different model families.
arXiv:2609.14896v1 Announce Type: cross Abstract: A notable byproduct of LLM alignment training is mode collapse: the progressive loss of output diversity that narrows a model's expressivity at infer...
arXiv:2604. 26170v2 Announce Type: replace Abstract: Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.
The paper introduces Self‑Routing, a post‑training framework that tailors optimization for each sample based on its rollout correctness and confidence. Instead of applying a single recipe to all data, samples are routed to different strategies—GRPO, on‑policy self‑distillation, regularization, or skipped—allowing training to adapt without external teachers or extra annotations. Experiments on Qwen3 and Qwen3.5 show consistent improvements over uniform methods and reveal that the routing distribution evolves during training, reducing unnecessary updates on low‑signal or already stable samples.
arXiv:2608. 09351v1 Announce Type: cross Abstract: Test-time scaling improves LLM accuracy but multiplies inference cost, making the accuracy gained per unit of compute the metric that matters in deployment.
arXiv:2505. 20161v2 Announce Type: replace-cross Abstract: Effective generalization in language models depends critically on the diversity of their training data.
arXiv:2601. 07055v2 Announce Type: replace Abstract: As high-quality data becomes increasingly difficult to obtain, self-evolution without curated training data has emerged as a promising paradigm.