Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,685 stories · RSS feed

arXiv Machine Learning

Oct 15

Reinforcement Learning via Advantage Guidance

arXiv:2610.19783v1 Announce Type: new Abstract: Reinforcement learning (RL) holds the promise of learning a policy that can outperform what is shown in the data. However, in practice, such systems su...

By Zhiyuan Zhou, Sergey Levine

diffusion reinforcement-learning benchmarks

arXiv Machine Learning

Oct 15

Rent-Seeking Language Models: Reward Models Set the Prices, Frequent Audits Curb the Rent

arXiv:2610.19832v1 Announce Type: new Abstract: Reward model overoptimization in RLHF is usually described as reward hacking. We recast it as rent-seeking: the policy, a firm, earns reward from a reg...

By Zixiao Wang, Xingbo Du, Duzhen Zhang, Le Song

llms reinforcement-learning

arXiv Machine Learning

Oct 15

Long-Horizon RL Needs Muon

arXiv:2610.20256v1 Announce Type: new Abstract: Long-horizon agents must coordinate reasoning, tool use, and environment feedback over extended interactions. Learning from sparse outcome rewards requ...

By Kai Ruan, Jinghao Lin, Tao Chen, Ziqi Zhou, Qianshan Wei, Zihe Huang

agents reinforcement-learning

arXiv Machine Learning

Oct 15

Hyperspherical Contrastive Representations Unlock Planning and Exploration

arXiv:2610.20409v1 Announce Type: new Abstract: We develop a self-supervised reinforcement learning algorithm that learns to build towers of up to eight cubes from scratch, without prior data, reward...

By Jonghae Park, Sungil Son, Jonghun Shin, Junseo Lee, H. Jin Kim, Benjamin Eysenbach

reinforcement-learning robotics

arXiv Machine Learning

Oct 15

Average-Reward Policy Identification from a Single Continuing Trajectory

arXiv:2610.20486v1 Announce Type: new Abstract: In reinforcement learning, an agent must explore an unknown environment to learn how to make good decisions. Unlike methods that can freely sample any...

By Tongxi Wang

agents reinforcement-learning

arXiv Machine Learning

Oct 15

Tail Competition Explains Scaling in Best-of-N Sampling for Verifiable Problems

arXiv:2610.19362v1 Announce Type: cross Abstract: Best-of-N sampling is a simple inference-time scaling strategy for large language models, yet its behavior under incomplete reward models remains poo...

By Taro Yano, Yoichi Ishibashi, Masafumi Oyamada

llms reinforcement-learning

arXiv Machine Learning

Oct 15

Explorer-Zero: Improving Self-Play for Long-Horizon Tasks with Decomposable Verifiable Rewards

arXiv:2610.19868v1 Announce Type: cross Abstract: Training LLM agents on long-horizon interactive tasks with reinforcement learning typically relies on human-annotated tasks and reliable verification...

By Tianyu Liu, Davide Belli, Bence Major, Arash Behboodi, Tribhuvanesh Orekondy

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Oct 15

Lifted State Policy Optimization: Internalizing Self-Correction through Transition Geometry

arXiv:2610.20350v1 Announce Type: cross Abstract: Preference optimization ranks final answers and trajectory RL maximizes aggregate returns, yet neither credits the local change that turns a flawed r...

By Ren Zhuang, Ben Wang, Shuifa Sun

reinforcement-learning benchmarks

arXiv Machine Learning

Oct 15

PrismaCoder: Pushing the Boundaries of Open Multimodal Code Intelligence through Decomposed Rubric Synthesis

arXiv:2610.20538v1 Announce Type: cross Abstract: Multimodal code intelligence connects the symbolic world of programs with the visual world their execution creates. Interest in this capability is bu...

By Qiushi Sun, Jingyang Gong, Yang Liu, Qiaosheng Chen, Zhangyue Yin, Zixian Huang, Hanxu Hu, Jing Xiong, Fangzhi Xu, Haochen Tan, Haoli Bai, Lifeng Shang, Xiaoguang Li, Ben Kao, Qipeng Guo, Lingpeng Kong, Jiahui Gao

reinforcement-learning multimodal data-engineering

arXiv Machine Learning

Oct 15

The Invisible Leash: Why RLVR May or May Not Escape Its Origin

arXiv:2507.14843v5 Announce Type: replace Abstract: Recent advances highlight Reinforcement Learning with Verifiable Rewards (RLVR) as a promising approach to enhancing LLM capabilities. However, it...

By Fang Wu, Weihao Xuan, Danny Xing, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, Yejin Choi

llms reinforcement-learning benchmarks

arXiv Machine Learning

Oct 15

NePPO: Near-Potential Policy Optimization for General-Sum Multi-Agent Reinforcement Learning

arXiv:2603.06977v3 Announce Type: replace Abstract: Multi-agent reinforcement learning (MARL) is increasingly used to design learning-enabled agents that interact in shared environments. However, tra...

By Addison Kalanther, Sanika Bharvirkar, Shankar Sastry, Chinmay Maheshwari

agents reinforcement-learning

arXiv Machine Learning

Oct 15

Generative Actor-Critic with Soft Bridge Policies

arXiv:2605.08733v2 Announce Type: replace Abstract: Expressive generative policies such as diffusion and flow models are appealing for maximum-entropy (MaxEnt) online reinforcement learning because o...

By Ke He, Le He, Shunpu Tang, Yafei Wang, Lisheng Fan

diffusion reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Oct 15

Learning, Fast and Slow: Towards LLMs That Adapt Continually

arXiv:2605.12484v3 Announce Type: replace Abstract: Large language models (LLMs) are trained for downstream tasks by updating their parameters (e.g., via RL). However, updating model parameters force...

By Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal, Joseph E. Gonzalez, Matei Zaharia, Kurt Keutzer, Inderjit S Dhillon, Rishabh Agarwal, Devvrit Khatri

llms reinforcement-learning

arXiv Machine Learning

Oct 15

Who Sets the Pace of Judge Hacking? Judge Capability versus the Policy-Judge Gap

arXiv:2610.15778v2 Announce Type: replace Abstract: Under GRPO against a frozen, reference-free LLM judge, RL exploits the judge's false positives. Does the policy-judge capability gap or the judge i...

By Haosen Wang

llms reinforcement-learning

arXiv Machine Learning

Oct 15

DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning

arXiv:2610.16597v2 Announce Type: replace Abstract: Expressive flow policies can represent multiple high-value actions, but sampling them with few integration steps while retaining tractable action d...

By Ruslan Ageev, Huaze Tang, Runqing Miao, Jiarong Li, Wenbo Ding

reinforcement-learning robotics