Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,242 stories · RSS feed

arXiv Machine Learning

Jun 25

Bias Fitting to Mitigate Length Bias of Reward Model in RLHF

arXiv:2505. 12843v2 Announce Type: replace Abstract: Reinforcement Learning from Human Feedback (RLHF) relies on reward models to align large language models with human preferences.

By Kangwen Zhao, Jianfeng Cai, Jinhua Zhu, Ruopei Sun, Dongyun Xue, Wengang Zhou, Li Li, Houqiang Li

llms reinforcement-learning safety

arXiv Machine Learning

Jun 25

A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficients

arXiv:2606. 24999v1 Announce Type: new Abstract: High-dimensional partial differential equations (PDEs) with unknown coefficients arise widely in scientific machine learning, including continuous-time reinforcement learning, yet solving them efficiently in a data-driven way remains challenging.

By Yanwei Jia, Du Ouyang, Huy\^en Pham, Xun Yu Zhou

reinforcement-learning safety

arXiv Machine Learning

Jun 25

Towards Scalable Multi-Task Reinforcement Learning with Large Decision Models

arXiv:2606. 24962v1 Announce Type: new Abstract: Recent progress in large-scale sequence modeling has shown that a single model can learn useful representations across highly diverse data distributions.

By Thibaut Kulak

llms agents reinforcement-learning robotics multimodal

arXiv Machine Learning

Jun 25

The Hitchhiker's Guide to Agentic AI: From Foundations to Systems

arXiv:2606. 24937v1 Announce Type: cross Abstract: The Hitchhiker's Guide to Agentic AI is a comprehensive practitioner's reference for building autonomous AI systems.

By Haggai Roitman

llms rag agents reinforcement-learning fine-tuning efficiency safety

arXiv Machine Learning

Jun 25

FBOS-RL: Feedback-Driven Bi-Objective Synergistic Reinforcement Learning

arXiv:2605. 20256v2 Announce Type: replace Abstract: Reinforcement learning has become a cornerstone for aligning and unlocking the reasoning capabilities of large-scale models.

By Xikai Zhang, Yongzhi Li, Likang Xiao, Yingze Zhang, Yanhua Cheng, Quan Chen, Peng Jiang, Wenjun Wu, Liu Liu

reinforcement-learning

arXiv Machine Learning

Jun 25

Solving Markov Decision Processes with Future Information via MPC

arXiv:2606. 24991v1 Announce Type: cross Abstract: Model Predictive Control (MPC) is widely used in industrial and robotic systems for enforcing constraints and embedding domain knowledge through finite-horizon optimization-based planning.

By Shambhuraj Sawant, Akhil S Anand, Dirk Reinhardt, Sebastien Gros

rag reinforcement-learning robotics

arXiv Machine Learning

Jun 25

TurboMPC: Fast, Scalable, and Differentiable Model Predictive Control on the GPU

arXiv:2606. 24039v1 Announce Type: cross Abstract: Robotics increasingly relies on GPUs for parallel simulation, large-scale learning, and neural-network inference.

By Gabriel Bravo-Palacios, Jianghan Zhang, Zachary Pestrikov, Brian Plancher, Thomas Lew

reinforcement-learning robotics benchmarks

arXiv Machine Learning

Jun 25

Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning

arXiv:2505. 19532v2 Announce Type: replace Abstract: The current state-of-the-art backdoor attacks against Reinforcement Learning (RL) rely upon unrealistically permissive access models, that assume the attacker can read (or even write) the victim's policy parameters, observations, or rewards.

By Shijie Liu, Andrew C. Cullen, Paul Montague, Sarah Erfani, Benjamin I. P. Rubinstein

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Hierarchical Reinforcement Learning for Neural Network Compression (HiReLC): Pruning and Quantization

arXiv:2606. 26002v1 Announce Type: new Abstract: We present HiReLC, a hierarchical ensemble-reinforcement learning framework for automated joint quantization and structured pruning of deep neural networks.

By Kamar Hibatallah Baghdadi, Kawther Guoual Belhamidi, Sara Belhadj, Aissa Boulmerka, Nadir Farhi

llms agents computer-vision reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 25

Beyond One-Size-Fits-All: Diagnosis-Driven Online Reinforcement Learning with Offline Priors

arXiv:2606. 25527v1 Announce Type: new Abstract: Online reinforcement learning (RL) agents increasingly depend on knowledge acquired offline to achieve practical efficiency.

By Guozheng Ma, Lu Li, Zilin Wang, Pierre-Luc Bacon, Dacheng Tao

agents reinforcement-learning robotics multimodal benchmarks

arXiv Machine Learning

Jun 25

The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems

arXiv:2606. 26057v1 Announce Type: cross Abstract: AI agents are granted access to tools, APIs, and other infrastructure, making them active principals in those systems.

By Seth Dobrin, {\L}ukasz Chmiel

agents reinforcement-learning safety

arXiv Machine Learning

Jun 25

Learning with a Single Rollout via Monte Carlo Pass@k Critic

arXiv:2606. 25451v1 Announce Type: new Abstract: Estimating token-level advantages in reinforcement learning (RL) for language models remains challenging because scaling up episodic experience collection is expensive.

By Fengdi Che, Yang Liu, Lei Yu, Meng Cao, Tong Che, Rupam Mahmood, Dale Schuurmans

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Supervised Reinforcement Learning for the Coordination of Distributed Energy Resources

arXiv:2606. 24947v1 Announce Type: new Abstract: The increasing integration of distributed energy resources (DERs) is crucial for power system decarbonization, yet unlocking DERs' flexibility is challenged by their inherent uncertainties and modelling complexity.

By Haoyuan Deng, Yihong Zhou, Thomas Morstyn, Yi Wang

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 25

Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methods

arXiv:2606. 25335v1 Announce Type: new Abstract: Multi-Agent Reinforcement Learning (MARL) value factorization methods can suffer from a loss of plasticity, gradually failing to adapt when transferring to new task instances.

By Zhengzhu Liu, Zeming Gao, Haoyuan Qin, Jiawei Hu, Junhao Wu, Miao Zhu, Haipeng Zhang, Chennan Ma, Siqi Shen, Cheng Wang

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See

arXiv:2606. 25127v1 Announce Type: new Abstract: We investigate how reward design shapes the internal attention patterns of reinforcement learning agents trained for autonomous driving.

By Mohamed Benabdelouahad, Ahmed Djalal Hacini, Nadir Farhi, Aissa Boulmerka

agents reinforcement-learning robotics safety

arXiv Machine Learning

Jun 25

Bias-Controlled Primal-Dual Natural Actor-Critic: Optimal Rates for Constrained Multi-Objective Average-Reward RL

arXiv:2606. 25012v1 Announce Type: new Abstract: Many reinforcement learning (RL) problems in the infinite-horizon average-reward setting require optimizing multiple conflicting objectives while satisfying multiple safety constraints.

By Ankur Naskar, Swetha Ganesh, Vaneet Aggarwal

agents reinforcement-learning safety

arXiv Machine Learning

Jun 25

ExTra: Exploratory Trajectory Optimization for Language Model Reinforcement Learning

arXiv:2606. 24994v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) for language-model reasoning can fail at both extremes of task difficulty: easy prompts often produce all-correct, low-diversity rollout groups with little gradient signal, while hard prompts can produce all-incorrect groups with no positive reward.

By Wenyang Hu, Junxiang Jia, Zhen Shu, Daniel Dahlmeier, See-Kiong Ng, Bryan Kian Hsiang Low

llms rag reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

LLM Evolution as an Industry-Scale Ecosystem: A Lifecycle Perspective on Continual Learning

arXiv:2606. 24901v1 Announce Type: new Abstract: Continual learning capability is critical for Industrial LLMs, as deployed models must be continuously updated to meet evolving requirements and environments, rather than repeatedly retrained from scratch.

By Hao Jiang, Enneng Yang, Guojie Zhu, Yibin Chen, Yunkun Xu, Zifu Kou, Jiayi Li, Chong Chen, Zhao Cao, Li Shen

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Geo-Strat-RL: Learning Geological Event Reasoning from Verifiable Tasks

arXiv:2606. 25000v1 Announce Type: new Abstract: To evaluate whether vision-language models can reason about geological histories, it is necessary to construct observations for which the underlying process history is known.

By Lukas Mosser

llms reinforcement-learning multimodal

arXiv Machine Learning

Jun 25

The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms

arXiv:2606. 25450v1 Announce Type: new Abstract: Traditional evaluations measure a learning algorithm's final performance on an i.

By Jinghan Zhang, Zerui Cheng, Shiqi Chen, Ge Zhang, Wenhao Huang, Jiashuo Liu, Junxian He, Tianle Cai

reinforcement-learning fine-tuning efficiency benchmarks

Get the daily briefing