arXiv AI October 19, 2026 By Ziyun Zou, Jiwoo Hong, Yifan Lu, Shao Tang, Weiyi Wu, Aman Gupta, Jin Tian, Imran Razzak

Beyond Reward Scoring: Online Multi-Agent Preference Learning for Multimodal Response Generation

The paper introduces PrefMind, an online dual‑agent reinforcement learning framework that jointly trains a response Generator and a preference Critic to explicitly learn and apply human preference judgments. By having the agents discuss and correct each other’s evaluations of candidate responses, PrefMind embeds preference understanding directly into the generation process rather than relying solely on a scalar reward model. Experiments on models of varying sizes demonstrate gains in both preference judgment accuracy and downstream response quality.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv AI

Jun 9

A Regret Minimization Framework on Preference Learning in Large Language Models

arXiv:2606. 09124v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has enabled progress on reasoning-intensive tasks by relying on task-specific verifiers that provide automated correctness signals.

By Suhwan Kim, Taehyun Cho, Geon-Hyeong Kim, Yu Jin Kim, Youngsoo Jang, Moontae Lee, Jungwoo Lee

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 2

From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning

arXiv:2606. 01123v1 Announce Type: new Abstract: Preference-based reinforcement learning (PbRL) avoids explicit reward engineering by learning from pairwise human preference feedback.

By Jun-Jie Yang, Chia-Heng Hsu, Kui-Yuan Chen, Ping-Chun Hsieh

reinforcement-learning fine-tuning

arXiv Machine Learning

Aug 11

Multi-Agent Reinforcement Learning via Agent-Specific Preference

arXiv:2608. 08604v1 Announce Type: new Abstract: Multi-agent reinforcement learning (MARL) is a powerful framework for solving complex collaborative tasks, but it relies heavily on well-defined global reward functions.

By Ni Mu, Yao Luan, Yiqin Yang, Qing-Shan Jia

agents reinforcement-learning benchmarks

arXiv AI

Oct 15

RewardHarness: Self-Evolving Agentic Post-Training

arXiv:2605.08703v2 Announce Type: replace Abstract: Aligning generative models with human preferences remains a central challenge. While RLHF is widely adopted, it relies on large-scale preference da...

By Yuxuan Zhang, Cong Wei, Penghui Du, Bo Li, Junwen Miao, Huaisong Zhang, Songcheng Cai, Yubo Wang, Dongfu Jiang, Yuyu Zhang, Changqian Yu, Ping Nie, Wenhu Chen, Kelsey R. Allen

llms agents reinforcement-learning

arXiv AI

Sep 1

Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment

The paper introduces Personalized Group Relative Policy Optimization (P‑GRPO), a new alignment framework for large language models that separates advantage estimation from immediate batch statistics. By normalizing advantages using preference‑group‑specific reward histories instead of the concurrent generation group, P‑GRPO maintains contrastive signals for distinct user preferences. Experiments across various tasks show that P‑GRPO converges faster and yields higher rewards than standard GRPO, improving alignment with heterogeneous human preferences while preserving general capabilities.

By Jialu Wang, Heinrich Peters, Asad A. Butt, Navid Hashemi, Alireza Hashemi, Pouya M. Ghari, Joseph Hoover, James Rae, Morteza Dehghani

llms reinforcement-learning safety

arXiv AI

Jul 7

Adaptive Margin RLHF via Preference over Preferences

arXiv:2509. 22851v4 Announce Type: replace-cross Abstract: Margin-based optimization is fundamental to improving generalization and robustness in classification tasks.

By Yaswanth Chittepu, Prasann Singhal, Greg Durrett, Scott Niekum

reinforcement-learning safety

One email a morning, machine-written

Related stories

A Regret Minimization Framework on Preference Learning in Large Language Models

From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning

Multi-Agent Reinforcement Learning via Agent-Specific Preference

RewardHarness: Self-Evolving Agentic Post-Training

Personalized Group Relative Policy Optimization for Heterogenous Preference Alignment

Adaptive Margin RLHF via Preference over Preferences