Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,074 stories · RSS feed

arXiv Machine Learning

Jul 14

Nonparametric Bayesian Inverse Reinforcement Learning with Data-Parallel Gibbs Sampling

arXiv:2607. 09886v1 Announce Type: new Abstract: Inverse Reinforcement Learning recovers reward functions from expert demonstrations, but standard formulations assume that all demonstrations come from a single expert.

By Sai Anirudh Katupilla, Shreeya Dasa Lakshminath

reinforcement-learning efficiency

arXiv AI

Jul 14

Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning

arXiv:2506. 07040v4 Announce Type: replace-cross Abstract: We study model-free methods for distributionally robust infinite-horizon average-reward Markov decision processes (MDPs).

By Yang Xu, Swetha Ganesh, Vaneet Aggarwal

reinforcement-learning

arXiv Machine Learning

Jul 14

Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning

arXiv:2410. 11251v2 Announce Type: replace Abstract: A hallmark of intelligent agents is the ability to learn reusable skills purely from unsupervised interaction with the environment.

By Jiaheng Hu, Zizhao Wang, Peter Stone, Roberto Mart\'in-Mart\'in

agents reinforcement-learning

arXiv Machine Learning

Jul 14

Predictive Divergence Masks for LLM RL

arXiv:2607. 10848v1 Announce Type: new Abstract: Reinforcement learning for large language models (LLMs) typically relies on trust-region masks to stabilize off-policy updates.

By Xiangxin Zhou, Jiarui Yao, Penghui Qi, Bowen Ping, Jiaqi Tang, Haonan Wang, Tianyu Pang

llms reinforcement-learning

arXiv AI

Jul 14

Transformer-Guided Swarm Intelligence for Frugal Neural Architecture Search

arXiv:2607. 11826v1 Announce Type: cross Abstract: Neural Architecture Search (NAS) has automated the design of deep learning models but traditionally requires massive computational resources, often measured in thousands of GPU-days.

By Romain Amigon

llms reinforcement-learning efficiency

arXiv AI

Jul 14

GRASP: GRanularity-Aware Search Policy for Agentic RAG

arXiv:2607. 10463v1 Announce Type: new Abstract: Agentic retrieval-augmented generation (RAG) extends static RAG by allowing language models to iteratively reason, generate search queries, retrieve evidence, and predict answers.

By Varun Gandhi, Jaewook Lee, Shantanu Todmal, Franck Dernoncourt, Ryan Rossi, Zichao Wang, Andrew Lan

llms rag agents nlp reinforcement-learning benchmarks

arXiv AI

Jul 14

SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL

arXiv:2607. 11185v1 Announce Type: new Abstract: Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital workflows through visual perception and GUI execution.

By Bowen Lv, Xiao Liu, Yanyu Ren, Hanyu Lai, Bohao Jing, Hanchen Zhang, Yanxiao Zhao, Shuntian Yao, Jie Tang, Yuxiao Dong

agents computer-vision reinforcement-learning benchmarks

arXiv Machine Learning

Jul 14

Reinforcement Learning for Execution under Dynamic Fees in a Closed-Loop DEX Simulator

arXiv:2607. 10960v1 Announce Type: new Abstract: Trader-facing dynamic fees are increasingly proposed for automated market makers (AMMs), but historical data do not identify how order flow would respond: trader-facing fees do not vary, trader types are latent, and a replayed tape is not a sequential decision environment.

By Wen-Ting Wang

agents reinforcement-learning benchmarks

arXiv AI

Jul 14

Beyond Euclidean Clipping: Overcoming Exploration Collapse in LLM RL via Riemannian Isometric Policy Optimization

arXiv:2607. 10169v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become a dominant paradigm for enhancing LLMs' reasoning capabilities.

By Zhicheng Cai, Xinyuan Guo, Hanlin Wu, Mingxuan Wang, Wei-Ying Ma, Ya-Qin Zhang, Hao Zhou

llms reinforcement-learning benchmarks safety

arXiv AI

Jul 14

World Models as Adversaries: Multi-Agent Self-Play Fine-Tuning for Robust Motion Planning

arXiv:2607. 10630v1 Announce Type: cross Abstract: Robust motion planning in dense traffic requires autonomous vehicles to interact in rare and safety-critical scenarios that are underrepresented in naturalistic driving data.

By Tong Nie, Yuewen Mei, Junlin He, Yihong Tang, Jian Sun, Wei Ma

agents reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 14

From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States

arXiv:2607. 09842v1 Announce Type: new Abstract: We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state trajectories of four open-weight transformer language models spanning four post-training regimes: no training (Gemma-4-E4B base), multimodal RLHF (Gemma-4-E4B-it), RL distillation (DeepSeek-R1-Distill-Qwen-7B), and SFT (Qwen2.

By Jorge A. Castillo, Marco Torres Y\'evenes, Juan Carlos Lanas

llms reinforcement-learning fine-tuning efficiency multimodal

arXiv Machine Learning

Jul 14

Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization

arXiv:2510. 08233v3 Announce Type: replace Abstract: Diffusion large language models (dLLMs) are promising alternatives to autoregressive large language models (AR-LLMs), as they potentially allow higher inference throughput.

By Yuchen Zhu, Wei Guo, Jaemoo Choi, Petr Molodyk, Bo Yuan, Molei Tao, Yongxin Chen

llms diffusion reinforcement-learning fine-tuning efficiency benchmarks

arXiv AI

Jul 14

SVR-R1: Bootstrapping Multi-modal Reasoning with Self-verification in Reinforcement Learning

arXiv:2607. 10966v1 Announce Type: new Abstract: We introduce Self-Verified Reasoner (SVR-R1), a multi-turn RL framework that turns a model's own verification into a learning signal for multimodal reasoning.

By Mingyuan Wu, Jingcheng Yang, Shengyi Qian, Xudong Wang, Jize Jiang, Qifan Wang, Aashu Singh, Khoi Pham, Fei Liu, Zhaolun Su, Zhuokai Zhao, Klara Nahrstedt, Jianyu Wang, Hanchao Yu

reinforcement-learning multimodal benchmarks

arXiv AI

Jul 14

Learning from Local Walks on Dynamic Graphs with Bandit Feedback

arXiv:2607. 10571v1 Announce Type: cross Abstract: We study stochastic multi-armed bandits on dynamic graphs, where arms correspond to the vertices of a network with time-varying edges.

By Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni, Lijun Chen

reinforcement-learning safety

arXiv AI

Jul 14

RVN-Bench: A Benchmark for Reactive Visual Navigation

arXiv:2603. 03953v2 Announce Type: replace-cross Abstract: Safe visual navigation is critical for indoor mobile robots operating in cluttered environments.

By Jaewon Lee, Jaeseok Heo, Gunmin Lee, Howoong Jun, Jeongwoo Oh, Songhwai Oh

agents reinforcement-learning robotics benchmarks

arXiv AI

Jul 14

ARMOR: Stabilizing On-Policy LLM RL with Off-Policy Anchor Samples

arXiv:2607. 10481v1 Announce Type: cross Abstract: Reinforcement learning (RL) has significantly enhanced the reasoning capabilities of large language models (LLMs), yet the training process remains notoriously fragile.

By Kexin Huang, Junkang Wu, Jinda Lu, Shuo Yang, Chiyu Ma, Jiancan Wu, Xiang Wang, Xiangnan He, Guoyin Wang, Jingren Zhou

llms reinforcement-learning benchmarks

arXiv AI

Jul 14

Filtering Harmful Actions Isn't Enough: Phantom Transfer in Agentic SDF

arXiv:2607. 10750v1 Announce Type: new Abstract: Synthetic data is widely used to train large language models because it is inexpensive to generate and easy to control.

By Chinmayi Dixit

llms agents reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 14

Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies

arXiv:2607. 10991v1 Announce Type: cross Abstract: As mobile robots become more integrated into everyday human environments, social robot navigation is becoming essential for ensuring human comfort, safety, and trust.

By Ali Ahmadi, Hamed Rahimi, Adrien Jacquet Cretides, Marie Samson, Mahdi Khoramshahi, Mohamed Chetouani

llms reinforcement-learning robotics multimodal benchmarks safety

arXiv AI

Jul 14

Learning to Fine-tune Foundation Models under Resource Limitations

arXiv:2607. 10694v1 Announce Type: cross Abstract: We study the problem of optimal continual fine-tuning for a pre-trained Foundation Model deployed at a resource-limited device.

By Thomas Tsouparopoulos, Iordanis Koutsopoulos

nlp reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 14

Diversified Multinomial Logit Contextual Bandits

arXiv:2607. 11684v1 Announce Type: cross Abstract: Existing contextual multinomial logit (MNL) bandits model relevance-driven choice but ignore the potential benefits of within-assortment diversity, while submodular/combinatorial bandits encode diversity in rewards but lack structured choice probabilities.

By Heesang Ann, Taehyun Hwang, Min-hwan Oh

reinforcement-learning

Get the daily briefing