arXiv AI

Structure-aware Relative Policy Optimization for Ranking

arXiv:2607. 25268v1 Announce Type: cross Abstract: Ranking is a fundamental component of modern information access systems.

arXiv AI

Sep 30

RankBuffer: Efficient Ranking-Based Rewards for Open-Ended Generation

arXiv:2609.36652v1 Announce Type: new Abstract: Open-ended generation lacks canonical answers, making pointwise rewards difficult to calibrate for group-based reinforcement learning. Directly ranking...

By Zixuan Yang, Yiqun Chen, Qi Liu, Wei Yang, Erhan Zhang, Liyi Chen, Qimeng Wang, Yan Gao, Jiaxin Mao

reinforcement-learning efficiency benchmarks

Hugging Face Trending Papers

Sep 20

RLVR$^{2}$: Reinforcement Learning with Verifiable Rubric-based Ranking

Reinforcement Learning with Verifiable Rewards (RLVR) is expanding from tasks with well-defined correctness signals, such as mathematics and code, toward multifaceted quality requirements specified by...

reinforcement-learning benchmarks

arXiv Machine Learning

Aug 7

RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction

arXiv:2608. 06310v1 Announce Type: new Abstract: Recent advances in reward modeling show a paradigm shift from discriminative reward models to generative reward models.

By Chenglong Wang, Ziming Zhu, Yifu Huo, Bei Li, Qiaozhi He, Yan Ding, Xiaoyang Hao, Yuxin Gao, Tianhua Zhou, Xiaojia Chang, Tongran Liu, Jingbo Zhu

llms reinforcement-learning benchmarks

arXiv Machine Learning

Sep 30

Efficient Offline Learning of Ranking Policies via Top-$k$ Policy Decomposition

arXiv:2609.36740v1 Announce Type: new Abstract: Many recommender systems such as for e-commerce and news platforms aim to provide users with rankings they are likely to interact with. Off-Policy Lear...

By Ren Kishimoto, Koichi Tanaka, Haruka Kiyohara, Yusuke Narita, Yasuo Yamamoto, Nobuyuki Shimizu, Yuta Saito

reinforcement-learning safety

arXiv Machine Learning

Sep 22

RLVR$^{2}$: Reinforcement Learning with Verifiable Rubric-based Ranking

arXiv:2609.23457v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) is expanding from tasks with well-defined correctness signals, such as mathematics and code, towa...

By Hao Li, Zhengkun Zhang, Gangqiang Hu, Zhen Zhang, Yude Gao, Dai Dai, Jing Liu

reinforcement-learning benchmarks

arXiv AI

Jul 3

Optimizing RAG Rerankers with LLM Feedback via Reinforcement Learning

arXiv:2604. 02091v2 Announce Type: replace-cross Abstract: Rerankers play a pivotal role in refining retrieval results for Retrieval-Augmented Generation.

By Yuhang Wu, Xiangqing Shen, Fanfan Wang, Cangqi Zhou, Zhen Wu, Xinyu Dai, Rui Xia

llms rag reinforcement-learning benchmarks

arXiv Machine Learning

Aug 24

Reinforcing Multi-Turn Reasoning in LLM Agents via Fine-Grained Reward Structure and Credit Assignment

The paper explores how dense, turn-level reward structures can improve reinforcement learning for large language model agents in multi-turn tasks. It introduces three reward granularity types—terminal, delayed, and per-turn—and adapts Group Relative Policy Optimization and Proximal Policy Optimization to each. Experiments on search and game agents show that per-turn rewards consistently yield better training dynamics, faster convergence, and higher answer correctness compared to sparse terminal or delayed rewards.

By Quan Wei, Siliang Zeng, Chenliang Li, Zhongruo Wang, William Brown, Oana Frunza, Wei Deng, Anderson Schneider, Yuriy Nevmyvaka, Yang Katie Zhao, Alfredo Garcia, Mingyi Hong

llms agents reinforcement-learning benchmarks

arXiv AI

4d ago

The Reward Model Selection Crisis in Personalized Alignment

arXiv:2512.23067v3 Announce Type: replace Abstract: Personalized alignment from preference data has focused primarily on improving personal reward model (RM) accuracy, with the implicit assumption th...

By Fady Rezk, Yuangang Pan, Chuan-Sheng Foo, Xun Xu, Nancy Chen, Henry Gouk, Timothy Hospedales

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 23

Rewarding Better Thinking for LLM Preference Alignment

arXiv:2607. 19824v1 Announce Type: new Abstract: LLM preference alignment aims to optimize models toward human preferences across diverse user instructions.

By Xubo Liu, Wenya Guo, Ruxue Yan, Xinying Qian, Ying Zhang

llms reinforcement-learning benchmarks safety

arXiv AI

Jul 23

ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking

arXiv:2601. 06487v3 Announce Type: replace-cross Abstract: Reinforcement learning has substantially improved the performance of LLM agents on tasks with verifiable outcomes, but it still struggles on open-ended agent tasks with vast solution spaces (e.

By Qiang Zhang, Boli Chen, Fanrui Zhang, Ruixue Ding, Shihang Wang, Qiuchen Wang, Yinfeng Huang, Haonan Zhang, Rongxiang Zhu, Pengyong Wang, Ailin Ren, Xin Li, Pengjun Xie, Jiawei Liu, Ning Guo, Jingren Zhou, Zheng-Jun Zha

llms agents reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Aug 18

TRCA: Transition-wise Rubric Credit Assignment for Long-horizon LLM Agents

arXiv:2608. 16156v1 Announce Type: new Abstract: Long-horizon large language model (LLM) agents are typically optimized with sparse terminal outcomes, making fine-grained credit assignment across multi-step interactions difficult.

By Huan Zhang, Mingju Chen, Dongxu Zhou, Can Lv, Heng Chang, Sen Cui, Faguo Wu, Shiji Zhou

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 10

3SPO: State-Score-Supervised Policy Optimization for LLM Agents

arXiv:2606. 09961v1 Announce Type: cross Abstract: Training large language models (LLMs) as autonomous agents via reinforcement learning (RL) has enabled frontier models to achieve superhuman performance in long-horizon tasks.

By Yu Han, Kailing Li, Yang Jiao, Yulin Dai, Yuqian Fu, Linhai Zhuo, Tianwen Qian

llms agents reinforcement-learning

Related stories