arXiv Computation and Language

ToSCA: Leveraging Hierarchical Reinforcement Learning on Temporal and Strategic Abstractions of Conversational Agents

arXiv Computation and Language
Sep 10

SocialRL: Refining LLMs' Social Intelligence through Multi-turn Reinforcement Learning and Reward Design

SocialRL is a multi-turn reinforcement learning framework that refines large language models’ social intelligence. It uses PPO to propagate delayed outcome rewards across turns, enabling long‑horizon planning, and introduces six process reward dimensions—such as goal advancement and relational attunement—to capture the goal‑relationship trade‑off. A reward model provides fine‑grained scoring and a stage‑aware weight schedule prioritizes relationship building early, goal pursuit mid‑way, and balanced closure later, yielding an average 9.2 percentage‑point improvement in goal achievement across multiple social‑dialogue benchmarks.

By Jianing Wang, Xintao Wang, Aili Chen, Jie Shi, Hongcheng Guo, Jun Gao, Wenxuan Zhao, Chengkun Lang, Yuanli Guo, Yanghua Xiao
arXiv Machine Learning
Jul 21

Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR

arXiv:2509. 02522v3 Announce Type: replace-cross Abstract: Recent advances in Reinforcement Learning with Verifiable Rewards (RLVR) have empowered large language models (LLMs) to tackle challenging reasoning tasks such as mathematics and programming, however existing RLVR methods often suffer from sparse reward signals and unstable policy gradient updates inherent to RL-based approaches.

By Jiaming Li, Longze Chen, Ze Gong, Yukun Chen, Lu Wang, Wanwei He, Run Luo, Min Yang
arXiv AI
Jul 31

MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialogue

arXiv:2603. 06194v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) for large language models (LLMs) has shown strong performance in single-turn tasks, but extending it to multi-turn interaction remains challenging due to sparse rewards and poor per-turn credit assignment.

By Naifan Zhang, Ruihan Sun, Jinwei Su, Hengjie Yang, Zhengyuan Pan, Zhaohan Chen, Xiaofan Zhang
arXiv AI
Jun 10

TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning

arXiv:2606. 11119v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models.

By Heming Zou, Qi Wang, Yun Qu, Yuhang Jiang, Lizhou Cai, Yixiu Mao, Ru Peng, Xin Xu, Weijie Liu, Kai Yang, Saiyong Yang, Xiangyang Ji
arXiv Machine Learning
Aug 27

Learning to summarize user information for personalized reinforcement learning from human feedback

The paper introduces PLUS, a framework that uses reinforcement learning to generate text-based summaries of individual users’ preferences, characteristics, and past conversations. These summaries condition a reward model, allowing it to predict personalized response preferences and improving reward accuracy by 11–77 % over the standard Bradley‑Terry model. PLUS demonstrates robust performance with new users and topics, achieves a 25 % improvement over existing personalized RLHF techniques, and enables zero‑shot personalization for state‑of‑the‑art models like GPT‑4.

By Hyunji Nam, Yanming Wan, Mickel Liu, Peter Ahnn, Jianxun Lian, Natasha Jaques