arXiv Machine Learning October 12, 2026 By Emily Xia Meng, Xuanyu Yang, Junaid Hasan, Michael Ruofan Zeng, Henry Kvinge

Studying the Effects of Task Diversity Using Small Mathematical Worlds

Read the original on arXiv Machine Learning →

The paper introduces two new mathematical task ecosystems, IntWorld and PermWorld, that consist of related integer and permutation tasks. These procedurally generated datasets allow researchers to control the number of tasks and study how task diversity affects neural network training. Initial experiments suggest that greater task variety leads to richer internal representations in models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Sep 23

PermuFormer: Multi-Task Pretraining for Permutation Representation in Algebraic Combinatorics

arXiv:2609.25438v1 Announce Type: new Abstract: Diverse pretraining has been shown to be an effective method for learning reusable, domain-aware representations that provide a starting point for fine...

By Henry Kvinge

llms fine-tuning

arXiv Machine Learning

Jul 31

Epistemic diversity across language models mitigates knowledge collapse

arXiv:2512. 15011v3 Announce Type: replace Abstract: Artificial intelligence (AI) increasingly generates the very content used to train future AI systems.

By Damian Hodel, Jevin D. West

llms

arXiv Computation and Language

Oct 16

Training Reasoning Models to Generate Diverse Scientific Ideas

The paper introduces a method for training large language models to generate diverse, high‑quality scientific ideas grounded in input documents. By using a set‑level reinforcement learning objective that rewards ideas for being distinct from earlier ones, the authors train an 8B model that outperforms Claude Opus 4.8 and GPT‑6 Astra on a contamination‑free benchmark. Human experts rate the model’s idea sets as equally promising while containing slightly more unique ideas, and automated experiments show comparable executability.

By Manya Wadhwa, Pranav Narayanan Venkit, Hiroaki Hayashi, Yu Li, Tuhin Chakrabarty, Chien-Sheng Wu

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 2

"I've Seen How This Goes": Characterizing Diversity via Progressive Conditional Surprise

arXiv:2606. 01811v1 Announce Type: cross Abstract: Measuring the diversity of creative outputs is central to evaluating post-training mode collapse, comparing decoding strategies, and quantifying creative behavior in both AI and human writing.

By Matthew Khoriaty, David Williams-King, Shi Feng

llms rag reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 21

Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning

arXiv:2505. 20161v2 Announce Type: replace-cross Abstract: Effective generalization in language models depends critically on the diversity of their training data.

By Jaehun Jung, Seungju Han, Ximing Lu, Skyler Hallinan, David Acuna, Shrimai Prabhumoye, Mostafa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi

llms benchmarks

arXiv AI

Jul 1

Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Models

arXiv:2410. 12341v4 Announce Type: replace-cross Abstract: As AI-generated content increasingly populates the web, generative AI models are at growing risk of being trained on their own outputs, a process known as AI autophagy.

By Daniele Gambetta, Gizem Gezici, Fosca Giannotti, Dino Pedreschi, Alistair Knott, Luca Pappalardo

llms fine-tuning

One email a morning, machine-written

Related stories

PermuFormer: Multi-Task Pretraining for Permutation Representation in Algebraic Combinatorics

Epistemic diversity across language models mitigates knowledge collapse

Training Reasoning Models to Generate Diverse Scientific Ideas

"I've Seen How This Goes": Characterizing Diversity via Progressive Conditional Surprise

Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning

Learning by Surprise: Adaptive Mitigation of Model Collapse in Large Language Models