arXiv Machine Learning

EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning

EmoGavel is a reinforcement learning framework that incorporates verifiable rewards to improve multimodal emotion preference learning. It introduces grounded supervised fine‑tuning (Grounded SFT) to jointly learn preference prediction, error localization, repair, and missing‑evidence recovery, and then applies grounding‑aware group relative policy optimization (GA‑GRPO) to prioritize preference correctness and recover learning signals from all‑wrong groups. Experiments on EmoPrefer and controlled‑error evaluation sets demonstrate that EmoGavel enhances natural preference prediction, controlled‑error discrimination, and maintains strong error‑grounding performance.

arXiv AI

6d ago

Beyond Reward Scoring: Online Multi-Agent Preference Learning for Multimodal Response Generation

The paper introduces PrefMind, an online dual‑agent reinforcement learning framework that jointly trains a response Generator and a preference Critic to explicitly learn and apply human preference judgments. By having the agents discuss and correct each other’s evaluations of candidate responses, PrefMind embeds preference understanding directly into the generation process rather than relying solely on a scalar reward model. Experiments on models of varying sizes demonstrate gains in both preference judgment accuracy and downstream response quality.

By Ziyun Zou, Jiwoo Hong, Yifan Lu, Shao Tang, Weiyi Wu, Aman Gupta, Jin Tian, Imran Razzak

agents reinforcement-learning multimodal safety

arXiv AI

Aug 28

AffectOmni: RL-Verifiable People-Centric Grounded Affective Reasoning for Social and Art-Related Scenes

AffectOmni is a reinforcement‑learning‑trained framework that enhances multimodal large language models for affective reasoning in social and art‑related scenes. It introduces People Focus and Temporal Order rewards to prioritize people‑centric cues and structured reasoning, and uses within‑group comparative scoring for more discriminative rewards. A Thinking Summarizer converts rationales into executable evidence instructions, which are grounded into pixel‑level regions via SAM3, enabling external auditability.

By Yibo Wang, Rui Yang, Jisheng Dang, Bimei Wang, Yitao Wu, Pengfei Cao, Wencan Zhang, Hong Peng, Bin Hu, Tat-Seng Chua

llms reinforcement-learning multimodal

arXiv AI

Jun 9

A Regret Minimization Framework on Preference Learning in Large Language Models

arXiv:2606. 09124v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has enabled progress on reasoning-intensive tasks by relying on task-specific verifiers that provide automated correctness signals.

By Suhwan Kim, Taehyun Cho, Geon-Hyeong Kim, Yu Jin Kim, Youngsoo Jang, Moontae Lee, Jungwoo Lee

llms reinforcement-learning benchmarks

arXiv AI

Jul 24

EmoAgent-R1: Towards Multimodal Emotion Understanding with Reinforcement Learning-based Dynamic Agent Specialization

arXiv:2607. 21013v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved impressive performance in multimodal emotion recognition (MER) tasks and lifted MER to a new level that is complex emotion understanding with advanced video understanding abilities and natural language description.

By Lihuang Fang, Yuchen Zou, kebin Jin, Jinghui Qin

llms agents reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Jul 31

S-GRPO: Unified Post-Training for Large Vision-Language Models

arXiv:2604. 16557v2 Announce Type: replace Abstract: Current post-training methodologies for adapting Large Vision-Language Models (LVLMs) generally fall into two paradigms: Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL).

By Yuming Yan, Kai Tang, Sihong Chen, Ke Xu, Dan Hu, Qun Yu, Pengfei Hu

llms reinforcement-learning fine-tuning multimodal

arXiv AI

Jul 9

EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models

arXiv:2602. 23802v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable progress in visual reasoning and understanding tasks but still struggle to capture the complexity and subjectivity of human emotions.

By Yiyang Fang, Wenke Huang, Pei Fu, Yihao Yang, Kehua Su, Zhenbo Luo, Jian Luan, Mang Ye

llms reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Jun 2

Mitigating Perceptual Judgment Bias in Multimodal LLM-as-a-Judge via Perceptual Perturbation and Reward Modeling

arXiv:2606. 02578v1 Announce Type: cross Abstract: Recent multimodal large language models have demonstrated strong reasoning ability, yet their reliability as automated evaluators remains limited by a critical weakness: when visual evidence conflicts with textual cues, MLLM judges tend to reward plausible narratives over perceptually correct answers.

By Seojeong Park, Jiho Choi, Junyong Kang, Seonho Lee, Jaeyo Shin, Hyunjung Shim

llms reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 2

Value-Free Policy Optimization via Reward Partitioning

arXiv:2506. 13702v4 Announce Type: replace-cross Abstract: Single-trajectory preference optimization methods learn from datasets of ((prompt, response, reward)) tuples, offering a practical alternative to pairwise preference learning by directly leveraging scalar feedback.

By Bilal Faye, Hanane Azzag, Mustapha Lebbah

llms reinforcement-learning fine-tuning

arXiv AI

Jun 19

AAPA: Adversarially Anchored Preference Alignment for Post-Training of Large Language Models

arXiv:2509. 25148v2 Announce Type: replace Abstract: Post-training alignment of large language models often combines supervised fine-tuning (SFT) on expert demonstrations with reinforcement learning (RL) from preference or verifiable feedback.

By Faqiang Qian, Kang An, Weikun Zhang, Ziliang Wang, Xuhui Zheng, Liangjian Wen, Yong Dai, Mengya Gao, Yichao Wu

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 14

SVR-R1: Bootstrapping Multi-modal Reasoning with Self-verification in Reinforcement Learning

arXiv:2607. 10966v1 Announce Type: new Abstract: We introduce Self-Verified Reasoner (SVR-R1), a multi-turn RL framework that turns a model's own verification into a learning signal for multimodal reasoning.

By Mingyuan Wu, Jingcheng Yang, Shengyi Qian, Xudong Wang, Jize Jiang, Qifan Wang, Aashu Singh, Khoi Pham, Fei Liu, Zhaolun Su, Zhuokai Zhao, Klara Nahrstedt, Jianyu Wang, Hanchao Yu

reinforcement-learning multimodal benchmarks

arXiv AI

Jul 16

Groc-PO: Grounded Context Preference Optimization for Truthful Multimodal LLMs

arXiv:2607. 13712v1 Announce Type: cross Abstract: Despite the rapid progress of Multimodal Large Language Models (MLLMs), they still suffer from untruthfulness issues, such as visual hallucinations, content fabrication, and unfaithful reasoning, which substantially undermine their faithfulness and practical utility.

By Zhixiao Zheng, Zheren Fu, Zhiyuan Yao, Chunxiao Liu, Dongming Zhang, Zhendong Mao

llms reinforcement-learning multimodal safety

arXiv AI

Aug 3

Multimodal Reinforcement Learning with Adaptive Verifier for AI Agents

arXiv:2512. 03438v3 Announce Type: replace Abstract: Agentic reasoning models trained with multimodal reinforcement learning (MMRL) have become increasingly capable, yet they are almost universally optimized using sparse, outcome-based rewards computed based on the final answers.

By Reuben Tan, Baolin Peng, Zhengyuan Yang, Hao Cheng, Oier Mees, Theodore Zhao, Andrea Tupini, Isar Meijer, Qianhui Wu, Yuncong Yang, Lars Liden, Yu Gu, Sheng Zhang, Xiaodong Liu, Lijuan Wang, Marc Pollefeys, Yong Jae Lee, Jianfeng Gao

agents reinforcement-learning robotics fine-tuning multimodal benchmarks safety

Related stories