Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,688 stories · RSS feed

arXiv AI

Oct 5

Not Until the Evidence Says So: Teaching LLM Investigators When to Close a Case

arXiv:2610.03190v1 Announce Type: cross Abstract: Accident, defect and outage investigations end with a decision that ordinary question answering never faces: whether the evidence gathered so far is...

By Tingzhu Bi, Ping Wang, Meng Ma

llms nlp reinforcement-learning fine-tuning safety

arXiv AI

Oct 5

MetaRubric: Learning to Reward for Rubric-Based Reinforcement Learning

arXiv:2610.02824v1 Announce Type: new Abstract: Rubric-based reinforcement learning extends reward-driven optimization to open-ended tasks by assigning partial credit to individual response requireme...

By Yuxuan Fan, Jaehong Yoon

reinforcement-learning multimodal benchmarks

arXiv Machine Learning

Oct 5

Demonstration-Guided Observation Attacks on Black-Box Safe Reinforcement Learning Controllers for Robotic Systems

arXiv:2602.16543v2 Announce Type: replace Abstract: Safe reinforcement learning (Safe RL) learns robotic controllers that optimize task rewards under safety constraints, yet observation perturbations...

By Jialiang Fan, Shixiong Jiang, Mengyu Liu, Fanxin Kong

reinforcement-learning robotics safety

arXiv AI

Oct 5

Student-Guided Teacher Distillation for Efficient LLM Task Routing: Positioning Against Jev-Style System-1 Classifiers

arXiv:2610.02516v1 Announce Type: cross Abstract: Zero-shot classifiers are useful for routing user requests to specialized LLM tasks, but scoring every request against a large candidate set is expen...

By Haifeng Wu, Srinivasan Manoharan, Jian Wan, Fangbo Tu, Junhua Zhao, Xin Chen

llms rag reinforcement-learning efficiency safety

arXiv AI

Oct 5

Instance-Dependent Regret for CMDPs with Step-Wise Constraints

arXiv:2610.02520v1 Announce Type: cross Abstract: We study online learning in episodic tabular constrained Markov decision processes with step-wise safety constraints. In such a setting, the constrai...

By Qian Zuo, Francesco Emanuele Stradi, Leyang Xue, Sattar Vakili

reinforcement-learning safety

arXiv Machine Learning

Oct 5

PerturbCellRL: Aligning Distributions and Grounding Biology via Post-Training Perturbation Generators

arXiv:2606.27752v2 Announce Type: replace Abstract: Single-cell perturbation models can reduce costly wet-lab screening by predicting how cells respond transcriptionally to interventions. Recent adva...

By Dongxia Wu, Mingyu Li, Yuhui Zhang, Anurendra Kumar, Emma Lundberg, Serena Yeung-Levy, Emily B. Fox

reinforcement-learning benchmarks safety

arXiv Machine Learning

Oct 5

Reward Inflation: A Healthy Stimulus for Reinforcement Learning

arXiv:2610.02545v1 Announce Type: new Abstract: Reward serves as the primary learning signal in reinforcement learning (RL). However, while reward magnitudes are typically held fixed throughout train...

By Ganghun Lee, Minji Kim, Minsu Lee, Byoung-Tak Zhang

reinforcement-learning

arXiv Computation and Language

Oct 5

Hint-Guided Diversified Policy Optimization for LLM Reasoning

arXiv:2606.03021v4 Announce Type: replace Abstract: Recent developments in Large Language Models (LLMs) have showcased impressive reasoning capabilities, with Reinforcement Learning with Verifiable R...

By Zhiyu Cao, Kaixin Wu, Mingjie Zhong, Peifeng Li, Can Ye, Qiaoming Zhu

llms reinforcement-learning

arXiv AI

Oct 5

Gains and Collapse in On-Policy Distillation:A Reinforcement Learning Perspective

arXiv:2610.03185v1 Announce Type: new Abstract: On-policy distillation (OPD) has become an important approach to language model post-training. However, despite its performance gains, OPD can also col...

By Han Cui, Jianhao Yan, Yun Luo, Hongbo Zhang, Zhizhang Fu, Yue Zhang

llms reinforcement-learning fine-tuning efficiency

arXiv Machine Learning

Oct 5

Can SGD Handle Heavy-Tailed Noise?

arXiv:2508.04860v2 Announce Type: replace-cross Abstract: Stochastic Gradient Descent (SGD) is a cornerstone of large-scale optimization, yet its theoretical behavior under heavy-tailed noise -- comm...

By Ilyas Fatkhullin, Florian H\"ubler, Guanghui Lan

reinforcement-learning

arXiv Machine Learning

Oct 5

Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitions

arXiv:2604.14398v2 Announce Type: replace-cross Abstract: Rotating detonation engines (RDEs) are a promising propulsion concept that may offer higher thermodynamic efficiency and specific impulse tha...

By Kristian Holme, Jean Rabault, Ricardo Vinuesa, Mikael Mortensen

agents reinforcement-learning

arXiv AI

Oct 5

FSPO: Policy-Consistent Risk and Pareto-Feasible Control for Budgeted LLM RL Post-Training

arXiv:2610.02828v1 Announce Type: new Abstract: Adaptive LLM reinforcement-learning post-training changes multiple training actuators online, including rollout temperature, group size, clipping, KL r...

By Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Daren Zha, Jun Xiao

llms reinforcement-learning benchmarks

arXiv AI

Oct 5

Single or Multiple Policies for Phase-Structured Reinforcement Learning?

arXiv:2610.03475v1 Announce Type: cross Abstract: Many reinforcement-learning (RL) problems are non-stationary yet structured and can be decomposed into phases, each with its own transition probabili...

By Guilhem Loussouarn, Nancy Nayak, Kin K. Leung

reinforcement-learning

arXiv AI

Oct 5

Learning Low-Frequency Motion Control for Robust and Dynamic Robot Locomotion

arXiv:2209.14887v3 Announce Type: replace-cross Abstract: Robotic locomotion is often approached with the goal of maximizing robustness and reactivity by increasing motion control frequency. We chall...

By Siddhant Gangapurwala, Luigi Campanaro, Ioannis Havoutis

reinforcement-learning robotics

arXiv AI

Oct 5

Recursive Agent Optimization

arXiv:2605.06639v2 Announce Type: replace-cross Abstract: We introduce Recursive Agent Optimization (RAO), a reinforcement learning approach for training recursive agents: agents that can spawn and d...

By Apurva Gandhi, Satyaki Chakraborty, Xiangjun Wang, Aviral Kumar, Graham Neubig

llms agents reinforcement-learning

arXiv AI

Oct 5

What Does a ProcGen Generalization Gap Measure? Action Rules, Residual Entropy, and the Missing Random Floor

arXiv:2609.32532v2 Announce Type: replace-cross Abstract: A generalization gap in reinforcement learning, return on training levels minus return on held-out levels, is usually reported without a refe...

By Abhisek Keshari

reinforcement-learning

arXiv Computation and Language

Oct 5

Text-Centric Post-Training for Omni-Modal Reasoning

arXiv:2610.02819v1 Announce Type: new Abstract: Improving joint audio-visual reasoning in Omni Large Language Models typically incurs substantial data construction and training costs. Our diagnostics...

By Ziyang Cheng, Yuhao Wang, Hongcheng Liu, Qimin Wu, Jingru Fan, Chen Qian, Yanfeng Wang, Yu Wang

llms reinforcement-learning fine-tuning multimodal

arXiv AI

Oct 5

Credit Where It Matters: Dependency-Aware Policy Optimization for Terminal Agents

arXiv:2610.03634v1 Announce Type: new Abstract: Terminal-using agents benefit from reinforcement learning (RL) in coding, debugging, and other multi-step terminal tasks. In these tasks, later command...

By Yu Li, Guangfeng Cai, Long-Fei Li, Shuo Han, Shengtian Yang, Han Luo, Kaibing Yang, Lei Feng

agents reinforcement-learning benchmarks

arXiv AI

Oct 5

EyeRobot 2.0: Active Gaze for Precise Manipulation without Wrist Cameras

arXiv:2610.03710v1 Announce Type: cross Abstract: Inspired by human vision, we introduce a framework using active gaze to enable fine-grained bimanual manipulation with only a single stereo camera. E...

By Kush Hari, Justin Kerr, Nidhya Shivakumar, Samarth Mahapatra, Carmelo Sferrazza, Jiahui Lei, Jitendra Malik, C. Karen Liu, Ken Goldberg, Angjoo Kanazawa

reinforcement-learning robotics

arXiv AI

Oct 5

$T^5$: Twin-Critic Training for Token-Level Thoughts in Reinforcement Mid-Training

arXiv:2609.32791v2 Announce Type: replace Abstract: Reinforcement mid-training lets language models learn internal thoughts from unlabeled text, but efficient token-level credit assignment remains ch...

By Nan Qiao, Yebin Yang, Weinong Wang, Shuning Wang, Shangpin Peng, Fengyuan Lu, Xinming Wang, Zhehan Kan, Ruixu Zhang, Songyang Zhang, Sheng Yue, Yonglong Tian, Ju Ren

llms reinforcement-learning benchmarks

Get the daily briefing