Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,074 stories · RSS feed

arXiv AI

Jul 13

Transformer-Empowered Actor-Critic Reinforcement Learning for Sequence-Aware Service Function Chain Partitioning

arXiv:2504. 18902v3 Announce Type: replace-cross Abstract: In the forthcoming era of 6G networks, characterized by unprecedented data rates, ultra-low latency, and ubiquitous connectivity, effective management of Virtualized Network Functions (VNFs) is essential.

By Cyril Shih-Huan Hsu, Anestis Dalgkitsis, Paola Grosso, Chrysa Papagianni

llms reinforcement-learning benchmarks

arXiv AI

Jul 13

A Descriptive and Normative Theory of Human Beliefs in RLHF

arXiv:2506. 01692v2 Announce Type: replace Abstract: Human preferences in RLHF are typically modeled as a function of the human's reward function or corresponding optimal state-action values.

By Sylee Dandekar, Shripad Deshmukh, Frank Chiu, W. Bradley Knox, Scott Niekum

agents reinforcement-learning

arXiv AI

Jul 13

Preference Conditioned Multi-Objective Reinforcement Learning: Decomposed, Diversity-Driven Policy Optimization

arXiv:2602. 07764v2 Announce Type: replace-cross Abstract: Multi-objective reinforcement learning (MORL) seeks to train agents capable of balancing conflicting objectives.

By Tanmay Ambadkar, Sourav Panda, Shreyash Kale, Jonathan Dodge, Abhinav Verma

agents reinforcement-learning benchmarks

arXiv Machine Learning

Jul 13

Learning More from Less: Reinforcement Learning from Hindsight

arXiv:2607. 09042v1 Announce Type: new Abstract: Reinforcement learning (RL) is increasingly used to post-train vision-language-action (VLA) models, but every update consumes robot rollouts that are slow and costly to collect, making sample efficiency a central concern.

By Iris Xu, Sunshine Jiang, John Marangola, Nitish Dashora, Richard Li, Thomas Liu, Zexue He, Yuheng Zhi, Alex Pentland, Pulkit Agrawal, Zhang-Wei Hong

llms reinforcement-learning robotics multimodal

arXiv AI

Jul 13

ECHO: Prune To Act, Trace To Learn With Selective Turn Memory In Agentic RL

arXiv:2606. 31650v2 Announce Type: replace-cross Abstract: Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows.

By Zijun Xie, Binbin Zheng, Enlei Gong, Jihua Liu, Yuyang You, Lingfeng Liu, Jiayao Tang, Guanqun Zhao, Aoqi Hu, Zeyu Chen

agents reinforcement-learning benchmarks

arXiv AI

Jul 13

SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generation

arXiv:2607. 08983v1 Announce Type: cross Abstract: While autonomous coding agents have significantly advanced automated test generation, they remain fundamentally limited by lazy generation, a phenomenon where agents prematurely terminate tasks and systematically avoid complex programmatic logic, resulting in inadequate code coverage.

By Sijia Gu, Noor Nashid, Ali Mesbah

llms agents reinforcement-learning benchmarks

arXiv AI

Jul 13

Contrastive Weak-to-strong Generalization

arXiv:2510. 07884v2 Announce Type: replace-cross Abstract: Weak-to-strong generalization provides a promising paradigm for scaling large language models (LLMs) by training stronger models on samples from aligned weaker ones, without requiring human feedback or explicit reward modeling.

By Houcheng Jiang, Junfeng Fang, Jiaxin Wu, Tianyu Zhang, Chen Gao, Xiang Wang, Xiangnan He, Yang Deng

llms reinforcement-learning safety

arXiv AI

Jul 13

Communication-Efficient Digital-Twin Coordination for Heterogeneous LLM Embodied Agents over Computing Power Networks

arXiv:2607. 09330v1 Announce Type: new Abstract: Embodied agent teams powered by heterogeneous large language models (LLMs) are being widely deployed in physical artificial intelligence such as smart factories, warehouses, and service robotics.

By Nuocheng Yang, Sihua Wang, Zihan Chen, Tony Q. S. Quek, Changchuan Yin

llms agents reinforcement-learning robotics

arXiv AI

Jul 13

ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning

arXiv:2512. 16861v2 Announce Type: replace-cross Abstract: Long-horizon manipulation has been a long-standing challenge in the robotics community.

By Zihan Zhou, Animesh Garg, Ajay Mandlekar, Caelan Garrett

reinforcement-learning robotics fine-tuning benchmarks

arXiv AI

Jul 13

Prompt-Driven Exploration

arXiv:2607. 08837v1 Announce Type: cross Abstract: Exploration is essential to RL since a policy cannot improve by repeatedly sampling the behaviors it already prefers.

By Sunshine Jiang, John Marangola, David Zhang, Raghuram Kowdeed, Ruiyang Luo, Nitish Dashora, Richard Li, Pulkit Agrawal, Zhang-Wei Hong

llms reinforcement-learning robotics multimodal

arXiv AI

Jul 13

Heterogeneous Information-Bottleneck Coordination Graphs for Multi-Agent Reinforcement Learning

arXiv:2605. 17393v2 Announce Type: replace Abstract: Coordination graphs are a central abstraction in cooperative multi-agent reinforcement learning (MARL), yet existing sparse-graph learners lack a theoretically grounded mechanism to decide which edges should exist and how much information each edge should carry.

By Wei Duan, Junyu Xuan, En Yu, Xiaoyu Yang, Jie Lu

agents reinforcement-learning

arXiv Machine Learning

Jul 13

Policy-based Tuning of Autoregressive Image Models with Instance- and Distribution-Level Rewards

arXiv:2603. 23086v2 Announce Type: replace Abstract: Autoregressive (AR) models are highly effective for image generation, yet their standard maximum-likelihood estimation training lacks direct optimization for sample quality and diversity.

By Orhun Bugra Baran, Melih Kandemir, Ramazan Gokberk Cinbis

diffusion reinforcement-learning

arXiv AI

Jul 13

HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMs

arXiv:2604. 20140v2 Announce Type: replace Abstract: Direct Preference Optimization (DPO) is an effective framework for aligning large language models with human preferences, but it struggles with complex reasoning tasks.

By Darsh Kachroo, Arjun Prasaath Anbazhagan, Adriana Caraeni, Brennan Lagasse, Kevin Zhu

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 13

AlphaZero in Sparsely Rewarded Games: Limits and Auxiliary Supervision

arXiv:2607. 08984v1 Announce Type: cross Abstract: AlphaZero has demonstrated that a neural-guided Monte Carlo Tree Search can achieve superhuman performance, but strong play does not necessarily imply perfect play.

By Brent Kong, Tejas Ram, Tony Yue Yu

reinforcement-learning

arXiv Machine Learning

Jul 13

SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions

arXiv:2607. 08925v1 Announce Type: new Abstract: Training reinforcement-learning agents directly on physical robots makes every fall costly, since a fall can damage the platform and cannot be undone like a simulator reset; the goal is therefore to minimize falls during training rather than trade them off against return, as constrained Markov decision process (MDP) formulations do.

By Elham Daneshmand, Majid Khadiv, Glen Berseth, Hsiu-Chin Lin

agents reinforcement-learning robotics benchmarks safety

arXiv AI

Jul 13

Risk-Aware General-Utility Markov Decision Processes

arXiv:2607. 09298v1 Announce Type: cross Abstract: We study general-utility Markov decision processes (GUMDPs) with risk-aware objectives.

By Pedro P. Santos, F\'abio Vital, Alberto Sardinha, Francisco S. Melo

agents reinforcement-learning

arXiv Machine Learning

Jul 13

FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space

arXiv:2607. 08877v1 Announce Type: cross Abstract: Pretrained generative robot policies based on flow matching and diffusion have achieved impressive results across a wide range of manipulation tasks.

By Michael Murray, Daphne Chen, Simran Bagaria, Dean Fortier, Tess Hellebrekers, Galen Mullins, Harshavardhan Gajarla, Oier Mees, Maya Cakmak, Andrey Kolobov

diffusion reinforcement-learning robotics fine-tuning

arXiv Machine Learning

Jul 13

Upper-Linearizability of Online Non-Monotone DR-Submodular Maximization over Down-Closed Convex Sets

arXiv:2602. 20578v2 Announce Type: replace Abstract: We study online maximization of non-monotone Diminishing-Return(DR)-submodular functions over down-closed convex sets, a regime where existing projection-free online methods suffer from suboptimal regret and limited feedback guarantees.

By Yiyang Lu, Haresh Jadav, Mohammad Pedramfar, Ranveer Singh, Vaneet Aggarwal

reinforcement-learning

arXiv AI

Jul 13

Multimodal Reward Hacking in Reinforcement Learning

arXiv:2607. 09492v1 Announce Type: new Abstract: Reinforcement learning (RL) is increasingly used to align multimodal large language models (MLLMs), but higher rewards do not always imply better task performance.

By Jiayu Yao, Yiwei Wang, Anmeng Zhang, Zhe Sun, Songsong Wang, Lingrui Mei, Yuyao Ge, Shenghua Liu

llms reinforcement-learning fine-tuning multimodal safety

arXiv AI

Jul 13

Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing

arXiv:2607. 09015v1 Announce Type: cross Abstract: We study contextual bandit problems with correlated arms and access to surrogate reward signals produced by a machine learning model, motivated by applications such as large language model (LLM) routing.

By Ajay Narayanan Sridhar, Ronak Singh, Mehrdad Mahdavi, Vijaykrishnan Narayanan

llms reinforcement-learning benchmarks

Get the daily briefing