Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,242 stories · RSS feed

arXiv Machine Learning

Jun 25

Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning

arXiv:2505. 19532v2 Announce Type: replace Abstract: The current state-of-the-art backdoor attacks against Reinforcement Learning (RL) rely upon unrealistically permissive access models, that assume the attacker can read (or even write) the victim's policy parameters, observations, or rewards.

By Shijie Liu, Andrew C. Cullen, Paul Montague, Sarah Erfani, Benjamin I. P. Rubinstein

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Geo-Strat-RL: Learning Geological Event Reasoning from Verifiable Tasks

arXiv:2606. 25000v1 Announce Type: new Abstract: To evaluate whether vision-language models can reason about geological histories, it is necessary to construct observations for which the underlying process history is known.

By Lukas Mosser

llms reinforcement-learning multimodal

arXiv Machine Learning

Jun 25

SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning

arXiv:2507. 16518v3 Announce Type: replace-cross Abstract: Recent advances in multimodal large language models (MLLMs) have shown impressive reasoning capabilities.

By Xiuwei Chen, Wentao Hu, Hanhui Li, Yongxin Wang Jun Zhou, Zisheng Chen, Meng Cao, Yihan Zeng, Kui Zhang, Yu-Jie Yuan, Jianhua Han, Hang Xu, Xiaodan Liang

llms reinforcement-learning fine-tuning multimodal benchmarks

arXiv Machine Learning

Jun 25

Minimax PAC Bounds for Learning in Exogenous Contextual MDPs

arXiv:2606. 25170v1 Announce Type: cross Abstract: We study PAC learning in tabular discounted Markov decision processes with exogenous i.

By Corentin Pla, Hugo Richard, Marc Abeille, Vianney Perchet

agents reinforcement-learning

arXiv Machine Learning

Jun 25

FactorLibrary: From Polynomials to Circuits via Recursive Subgoals

arXiv:2606. 25394v1 Announce Type: new Abstract: Finding minimal arithmetic circuits for polynomials over finite fields is a combinatorially hard problem central to algebraic complexity theory.

By Rohan Pandey, Michael Ruofan Zeng, Weikun K. Zhang, Kaijie Jin, Naomi Morato, Archit Ganapule, Bhaumik Mehta, Jarod Alper

agents reinforcement-learning

arXiv Machine Learning

Jun 25

FBOS-RL: Feedback-Driven Bi-Objective Synergistic Reinforcement Learning

arXiv:2605. 20256v2 Announce Type: replace Abstract: Reinforcement learning has become a cornerstone for aligning and unlocking the reasoning capabilities of large-scale models.

By Xikai Zhang, Yongzhi Li, Likang Xiao, Yingze Zhang, Yanhua Cheng, Quan Chen, Peng Jiang, Wenjun Wu, Liu Liu

reinforcement-learning

arXiv Machine Learning

Jun 25

Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methods

arXiv:2606. 25335v1 Announce Type: new Abstract: Multi-Agent Reinforcement Learning (MARL) value factorization methods can suffer from a loss of plasticity, gradually failing to adapt when transferring to new task instances.

By Zhengzhu Liu, Zeming Gao, Haoyuan Qin, Jiawei Hu, Junhao Wu, Miao Zhu, Haipeng Zhang, Chennan Ma, Siqi Shen, Cheng Wang

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Bias-Controlled Primal-Dual Natural Actor-Critic: Optimal Rates for Constrained Multi-Objective Average-Reward RL

arXiv:2606. 25012v1 Announce Type: new Abstract: Many reinforcement learning (RL) problems in the infinite-horizon average-reward setting require optimizing multiple conflicting objectives while satisfying multiple safety constraints.

By Ankur Naskar, Swetha Ganesh, Vaneet Aggarwal

agents reinforcement-learning safety

arXiv Machine Learning

Jun 25

Safe Learning Control with Optimality and Stability Guarantees

arXiv:2501. 15373v2 Announce Type: replace-cross Abstract: Merely pursuing performance may adversely affect safety, while a conservative policy for safe exploration will degrade the performance.

By Xinyang Wang, Hongwei Zhang, Shimin Wang, Wei Xiao, Martin Guay

reinforcement-learning robotics safety

arXiv Machine Learning

Jun 25

Learning with a Single Rollout via Monte Carlo Pass@k Critic

arXiv:2606. 25451v1 Announce Type: new Abstract: Estimating token-level advantages in reinforcement learning (RL) for language models remains challenging because scaling up episodic experience collection is expensive.

By Fengdi Che, Yang Liu, Lei Yu, Meng Cao, Tong Che, Rupam Mahmood, Dale Schuurmans

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversity

arXiv:2606. 26091v1 Announce Type: new Abstract: On-policy self-distillation achieves strong pass@1 accuracy by using a single model as both teacher and student, with the teacher conditioned on a correct demonstration to provide dense token-level feedback.

By Andrei Liviu Nicolicioiu, Mohammad Pezeshki, Aaron Courville

reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jun 25

Neglected Free Lunch from Post-training: Progress Advantage for LLM Agents

arXiv:2606. 26080v1 Announce Type: new Abstract: Process reward models enable fine-grained, step-level evaluation of LLMs, yet building them for agentic settings remains prohibitively difficult: long-horizon interactions, irreversible actions, and stochastic environment feedback make both human annotation and Monte Carlo estimation infeasible at scale.

By Changdae Oh, Wendi Li, Seongheon Park, Samuel Yeh, Tanwi Mallick, Sharon Li

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

LLM Evolution as an Industry-Scale Ecosystem: A Lifecycle Perspective on Continual Learning

arXiv:2606. 24901v1 Announce Type: new Abstract: Continual learning capability is critical for Industrial LLMs, as deployed models must be continuously updated to meet evolving requirements and environments, rather than repeatedly retrained from scratch.

By Hao Jiang, Enneng Yang, Guojie Zhu, Yibin Chen, Yunkun Xu, Zifu Kou, Jiayi Li, Chong Chen, Zhao Cao, Li Shen

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jun 25

Low-Complexity Policy Tessellations in Structured Markov Decision Processes

arXiv:2606. 25593v1 Announce Type: new Abstract: We study optimal-policy geometry in structured Markov decision processes.

By Fredy Pokou (CRIStAL)

reinforcement-learning

arXiv Machine Learning

Jun 25

Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion

arXiv:2606. 25097v1 Announce Type: new Abstract: Speculative decoding accelerates inference by letting a draft model propose tokens for a target model to verify, raising a concrete safety question: at temperature zero, can draft-side behavior leak into safety-scored outputs?

By Sahil Kadadekar

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 25

A Zeroth-Order Deep Learning Method for Fully Nonlinear Parabolic Partial Differential Equations with Unknown Coefficients

arXiv:2606. 24999v1 Announce Type: new Abstract: High-dimensional partial differential equations (PDEs) with unknown coefficients arise widely in scientific machine learning, including continuous-time reinforcement learning, yet solving them efficiently in a data-driven way remains challenging.

By Yanwei Jia, Du Ouyang, Huy\^en Pham, Xun Yu Zhou

reinforcement-learning safety

arXiv Machine Learning

Jun 25

Bias Fitting to Mitigate Length Bias of Reward Model in RLHF

arXiv:2505. 12843v2 Announce Type: replace Abstract: Reinforcement Learning from Human Feedback (RLHF) relies on reward models to align large language models with human preferences.

By Kangwen Zhao, Jianfeng Cai, Jinhua Zhu, Ruopei Sun, Dongyun Xue, Wengang Zhou, Li Li, Houqiang Li

llms reinforcement-learning safety

arXiv Machine Learning

Jun 25

The Hitchhiker's Guide to Agentic AI: From Foundations to Systems

arXiv:2606. 24937v1 Announce Type: cross Abstract: The Hitchhiker's Guide to Agentic AI is a comprehensive practitioner's reference for building autonomous AI systems.

By Haggai Roitman

llms rag agents reinforcement-learning fine-tuning efficiency safety

arXiv Machine Learning

Jun 25

Solving Markov Decision Processes with Future Information via MPC

arXiv:2606. 24991v1 Announce Type: cross Abstract: Model Predictive Control (MPC) is widely used in industrial and robotic systems for enforcing constraints and embedding domain knowledge through finite-horizon optimization-based planning.

By Shambhuraj Sawant, Akhil S Anand, Dirk Reinhardt, Sebastien Gros

rag reinforcement-learning robotics

arXiv Machine Learning

Jun 25

TurboMPC: Fast, Scalable, and Differentiable Model Predictive Control on the GPU

arXiv:2606. 24039v1 Announce Type: cross Abstract: Robotics increasingly relies on GPUs for parallel simulation, large-scale learning, and neural-network inference.

By Gabriel Bravo-Palacios, Jianghan Zhang, Zachary Pestrikov, Brian Plancher, Thomas Lew

reinforcement-learning robotics benchmarks

Get the daily briefing