arXiv:2608. 07460v1 Announce Type: cross Abstract: While post-training improves the capabilities of large language models (LLMs), it generally lowers their output diversity and creativity, negatively impacting tasks that explicitly require creativity (e.
The paper introduces a method for training large language models to generate diverse, high‑quality scientific ideas grounded in input documents. By using a set‑level reinforcement learning objective that rewards ideas for being distinct from earlier ones, the authors train an 8B model that outperforms Claude Opus 4.8 and GPT‑6 Astra on a contamination‑free benchmark. Human experts rate the model’s idea sets as equally promising while containing slightly more unique ideas, and automated experiments show comparable executability.
The paper introduces the Agentic Compositional Generalization hypothesis, suggesting that reinforcement learning (RL) primarily refines high‑level decision‑making behaviors that orchestrate pre‑trained low‑level skills, rather than teaching new domain‑specific skills from scratch. It proposes River, a training recipe that enhances reward quality by filtering low‑quality synthetic environments and adding process‑level behavior regularization. Using River, RL‑trained agents outperform other open‑source 8B models on four terminal‑agent benchmarks, achieving significant gains with fewer than 30% of the training environments.
arXiv:2610.06729v2 Announce Type: replace Abstract: Large language models (LLMs) can generate accurate responses, but these are void of diversity. We attempt to address this for the task of creative...
HARPO is a reinforcement learning framework that jointly optimizes faithfulness and creativity in language generation. It uses a Hallucination-Aware Generative Reward Model (HA‑GRM) to evaluate both faithfulness and writing quality, and a Selective Activation Mechanism (SAM) that applies writing rewards only to hallucination‑free outputs. Experiments on Qwen models show that HARPO improves faithfulness scores and reduces hallucination rates while boosting creative‑writing performance.
EliteReplay is a post‑training method that combines GRPO with a quality‑diversity archive to preserve diverse solutions in reinforcement learning. By collecting distinct strategies from a frozen base model before training and replaying them during training, the approach improves accuracy while maintaining a broader set of correct solutions. On text‑to‑SQL tasks, EliteReplay outperforms baselines, achieving higher pass@128 scores and up to 77% more distinct correct strategies, and it better handles ambiguous questions by covering multiple interpretations.
arXiv:2510. 01171v4 Announce Type: replace-cross Abstract: Post-training alignment often reduces LLM diversity, leading to a phenomenon known as mode collapse.
arXiv:2510. 11686v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) promises to expand the capabilities of language models, but it is unclear if current RL techniques promote the discovery of novel behaviors, or simply sharpen those already present in the base model.
arXiv:2610.18572v1 Announce Type: new Abstract: Automated post-training transforms pre-trained large language models (LLMs) into models capable of handling diverse domains by automatically constructi...
arXiv:2505. 09655v5 Announce Type: replace-cross Abstract: Post-training LLMs with Reinforcement Learning, specifically Group Relative Policy Optimization (GRPO), has emerged as a paradigm for enhancing mathematical reasoning.
arXiv:2610.09943v1 Announce Type: cross Abstract: Reinforcement learning (RL) fine-tuning improves vision-language-action (VLA) policies through closed-loop experience, yet generalization beyond the...
arXiv:2605.28360v3 Announce Type: replace Abstract: Automatic prompt optimization (APO) has driven significant gains in LLM-based agentic workflows. However, most existing methods treat each task's p...