← Back to all news
arXiv Computation and Language By Shourov Joarder, Diganta Sikdar, Ahsan Habib Akash, Binod Bhattarai, Prashnna Gyawali

Two is better than one: A Collapse-free Multi-Reward RLIF Training Framework

Read the original on arXiv Computation and Language →

The paper introduces a multi-reward reinforcement learning from internal feedback (RLIF) framework that splits the training signal into an answer-level reward from cluster voting and a completion-level reward from token-wise self-certainty. It uses GDPO-based normalization to balance reward scales and adds KL-Cov regularization to maintain exploration and avoid entropy collapse. Experiments on mathematical reasoning and code-generation tasks show the method improves stability and robustness compared to prior unsupervised RL approaches, approaching the performance of supervised RLVR methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv Machine Learning
Aug 12

Reinforcement Learning-based Semi-supervised Knowledge Distillation with LLM-as-a-Judge

arXiv:2604. 02621v2 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) substantially improves the reasoning capabilities of language models, but most existing RL fine-tuning approaches rely entirely on ground-truth verifiable rewards and thus labeled datasets with verifiable answers.

By Yiyang Shen, Lifu Tu, Weiran Wang
llmsreinforcement-learningfine-tuningefficiency
More like this →
arXiv AI
Aug 19

Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL

Co‑RL is a multi‑agent reinforcement learning framework that trains several decoupled models without shared parameters, using rewards generated by their peers. By increasing cohort diversity—through heterogeneous model families, varying sizes, and rephrased training samples—Co‑RL reduces self‑reinforcing feedback loops, preserves behavioral diversity, and prevents training collapse. Across both text‑only and multimodal benchmarks, Co‑RL outperforms base models and prior label‑free methods, achieving gains of 3.0‑8.6% on seven text benchmarks and 2.3‑7.2% on four multimodal benchmarks, while matching or surpassing supervised approaches without any ground‑truth labels.

By Yunhao Yang, Yuexin Bian, Yunjie Tian, Di Fu, Tianjin Huang, Yuanyuan Shi, Ziang Xiao, Nuno Vasconcelos, Yijiang Li
llmsagentsreinforcement-learningmultimodalbenchmarks
More like this →
arXiv AI
Jun 4

GeoMin: Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling

arXiv:2606. 04516v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) significantly advances LLM reasoning, yet it faces a dilemma: standard supervised scaling is throttled by high annotation costs, while unsupervised alternatives suffer from severe model collapse.

By Guangcheng Zhu, Shenzhi Yang, Haobo Wang, Xing Zheng, Yingfan MA, Xuening Feng, Zhongqi Chen, Kai Tang, Zhengqing Zang, Bowen Song, Weiqiang Wang, Gang Chen
llmsreinforcement-learning
More like this →