The paper introduces Trajectory-guided Joint Policy Optimization (TJPO), a reinforcement‑learning framework that explicitly encourages diversity in the trajectories of large‑language‑model agents. By defining task‑specific trajectory descriptors, TJPO measures and optimizes diversity as a set‑level function, avoiding population‑based training. Experiments on Sokoban and ALFWorld demonstrate that TJPO yields diverse, interpretable behaviors while preserving strong task performance.
By Huaiyu Fu, Heng Cao, Hao Wang, Jian Ya, Tao Chen
arXiv:2506. 01568v4 Announce Type: replace Abstract: Being able to solve a task in diverse ways makes agents more robust to task variations and less prone to local optima.
By Cornelius V. Braun, Sayantan Auddy, Marc Toussaint
arXiv:2604. 17244v2 Announce Type: replace-cross Abstract: Large language model (LLM) agents for sequential decision-making struggle to produce diverse outputs.
By Priya Gurjar, Md Farhan Ishmam, Kenneth Marino
arXiv:2501. 04426v2 Announce Type: replace-cross Abstract: Offline diversity maximization under imitation constraints can transform demonstration data into a set of distinct behavioral policies, improving robustness to distribution shift without additional environment interaction.
By Pavel Kolev, Marin Vlastelica, Georg Martius
The paper introduces RePPO, a method that adapts Proximal Policy Optimization to jointly optimize a compact population of diverse, high‑quality policies using a population‑dependent, dynamically shaped reward. RePPO achieves a strong quality‑diversity tradeoff on standard continuous‑control tasks and scales to larger policy and behavior spaces, maintaining quality while significantly increasing diversity in pixel‑based tasks. The authors also demonstrate RePPO’s applicability to fine‑tuning large language models, showing for the first time that quality‑diversity reinforcement learning can scale to this domain.
By Saeed Hedayatian, Harsh Shah, Celine Mneimneh, Stefanos Nikolaidis
arXiv:2509.15194v4 Announce Type: replace-cross
Abstract: Large language models (LLMs) can improve through reinforcement learning with verifiable rewards (RLVR), but many settings lack reference answ...
By Yujun Zhou, Zhenwen Liang, Haolin Liu, Wenhao Yu, Kishan Panaganti, Linfeng Song, Dian Yu, Xiangliang Zhang, Haitao Mi, Dong Yu
SwiftExplorer is a training‑free diffusion model alignment plugin that addresses two key issues in objective‑guided sampling: the loss of diversity due to strong directional bias and the inefficiency of constant guidance. It introduces an Inheritance‑Restart exploration mechanism to prevent early convergence and enhance the likelihood of high‑reward trajectories, while a Quality‑Efficiency arbitration mechanism removes incorrect signals and dynamically stops generation when optimal reward gain is achieved. Experiments show that SwiftExplorer improves preference, fidelity, diversity, and richness across multiple evaluation metrics.
By Renye Yan, Jikang Cheng, You Wu, Bojin Huang, Wei Peng, Zongwei Wang, Ling Liang, Yimao Cai
EliteReplay is a post‑training method that combines GRPO with a quality‑diversity archive to preserve diverse solutions in reinforcement learning. By collecting distinct strategies from a frozen base model before training and replaying them during training, the approach improves accuracy while maintaining a broader set of correct solutions. On text‑to‑SQL tasks, EliteReplay outperforms baselines, achieving higher pass@128 scores and up to 77% more distinct correct strategies, and it better handles ambiguous questions by covering multiple interpretations.
By Hanane Nour Moussa, Doeun Lee, Sachin Kumar
arXiv:2601. 12401v2 Announce Type: replace-cross Abstract: Reinforcement learning (RL) has emerged as a powerful paradigm for fine-tuning large-scale generative models, such as diffusion and flow models, to align with complex human preferences and user-specified tasks.
By Jinmei Liu, Haoru Li, Zhenhong Sun, Chaofeng Chen, Yatao Bian, Bo Wang, Daoyi Dong, Chunlin Chen, Zhi Wang
arXiv:2606. 03962v1 Announce Type: cross Abstract: Classical reinforcement learning (RL) typically seeks a deterministic policy that maximizes the expected sum of a scalar reward.
By Anthony GX-Chen, Ankit Anand, Gheorghe Comanici, Zaheer Abbas, Eser Ayg\"un, David Smalling, Shibl Mourad, Doina Precup, Andr\'e Barreto, Mark Rowland
arXiv:2608. 19684v1 Announce Type: new Abstract: Recent studies investigate how to leverage pre-collected datasets to improve the policy performance and sample efficiency of RL.
By Tanachai Anakewat, Takayuki Osa, Tatsuya Harada
arXiv:2605. 07724v2 Announce Type: replace-cross Abstract: Recursive retraining of generative models poses a critical representation challenge: when synthetic outputs are curated based on a fixed reward signal, the model tends to collapse onto a narrow set of outputs that over-optimize that objective.
By Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson, Lukasz Golab