Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,242 stories · RSS feed

arXiv AI

Jul 1

ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs

arXiv:2606. 31054v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are critically hampered by hallucination, generating content inconsistent with the provided image.

By Zhiyuan Yao, Zheren Fu, Zhixiao Zheng, Jiajun Li, Yi Tu, Zhendong Mao

llms diffusion reinforcement-learning multimodal benchmarks safety

arXiv AI

Jul 1

Freeform Preference Learning for Robotic Manipulation

arXiv:2606. 32027v1 Announce Type: cross Abstract: Reward design remains a central bottleneck for autonomous robot policy improvement, especially in long-horizon manipulation tasks where sparse success labels provide too little signal and binary preferences collapse many competing notions of quality into one ambiguous signal.

By Marcel Torne, Anubha Mahajan, Abhijnya Bhat, Chelsea Finn

agents computer-vision reinforcement-learning robotics safety

arXiv Machine Learning

Jul 1

Teaching Models to Teach Themselves: Reasoning at the Edge of Learnability

arXiv:2601. 18778v3 Announce Type: replace Abstract: RL methods for scaling large reasoning models stall on datasets with low initial success rates, and thus little training signal.

By Shobhita Sundaram, John Quan, Ariel Kwiatkowski, Kartik Ahuja, Yann Ollivier, Julia Kempe

llms reinforcement-learning benchmarks

arXiv AI

Jul 1

AETDICE: Unified Framework and Offline Optimization for Nonlinear Multi-Objective RL

arXiv:2606. 31178v1 Announce Type: cross Abstract: Optimizing nonlinear preferences in multi-objective reinforcement learning (MORL) is essential for capturing complex trade-offs like risk aversion or fairness.

By Woosung Kim, Youngjun Suh, Jinho Lee, Jongmin Lee, Byung-Jun Lee

reinforcement-learning safety

arXiv AI

Jul 1

Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs

arXiv:2606. 32032v1 Announce Type: cross Abstract: Metacognition is a critical component of intelligence that describes the ability to monitor and regulate one's own cognitive processes.

By Gabrielle Kaili-May Liu, Avi Caciularu, Gal Yona, Idan Szpektor, Arman Cohan

llms reinforcement-learning benchmarks safety

arXiv Machine Learning

Jul 1

Addressing Over-Refusal in LLMs with Competing Rewards

arXiv:2606. 31748v1 Announce Type: new Abstract: Safety training on language models often induces over-refusal: improved safety on harmful prompts at the cost of increased refusal on harmless ones.

By Taeyoun Kim, Aviral Kumar

llms reinforcement-learning safety

arXiv AI

Jul 1

An AI-Based Solution for Secure Service Provisioning in IoT

arXiv:2606. 30701v1 Announce Type: cross Abstract: As the Internet of Things (IoT) continues its rapid expansion, the attack surface grows accordingly, with emerging threats targeting smart objects and their interactions.

By Marco Arazzi, Mert Cihangiroglu, Serena Nicolazzo, Antonino Nocera, Vinod P

agents reinforcement-learning

arXiv AI

Jul 1

From Search to Synthesis: Training LLMs as Zero-Shot Workflow Generators

arXiv:2606. 30704v1 Announce Type: cross Abstract: Large language models (LLMs) excel across a wide range of tasks, yet their instance-specific solutions often lack the structural consistency needed for reliable deployment.

By Gan Luo, Zihan Qin, Bin Dong, Wotao Yin

llms nlp reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 1

Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs

arXiv:2606. 31413v1 Announce Type: new Abstract: Composing independently trained LoRA adapters into a single large language model is useful for multi-domain adaptation, especially when the original training data cannot be shared.

By Seyed Alireza Molavi, Zhan Su, Yan Hu, Peyman Sheikholharam Mashhadi, Stefan Byttner, Prayag Tiwari

llms reinforcement-learning fine-tuning benchmarks

arXiv AI

Jul 1

Compositional Concept-Based Neuron-Level Interpretability for Deep Reinforcement Learning

arXiv:2502. 00684v2 Announce Type: replace-cross Abstract: Deep reinforcement learning (DRL) has successfully addressed many complex control problems.

By Zeyu Jiang, Hai Huang, Xingquan Zuo

agents computer-vision reinforcement-learning safety

arXiv AI

Jul 1

ECHO: Prune to act, trace to learn with selective turn memory in agentic RL

arXiv:2606. 31650v1 Announce Type: cross Abstract: Long-horizon language agents must repeatedly interact with tools, accumulate evidence, and make decisions under bounded context windows.

By Zijun Xie, Binbin Zheng, Enlei Gong, Jihua Liu, Yuyang You, Lingfeng Liu, Jiayao Tang, Guanqun Zhao, Aoqi Hu, Zeyu Chen

agents reinforcement-learning benchmarks

arXiv AI

Jul 1

Lyapunov-Certified Direct Switching Theory for Q-Learning

arXiv:2604. 19569v4 Announce Type: replace-cross Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.

By Donghwan Lee

reinforcement-learning

arXiv Machine Learning

Jul 1

A Complete Characterization of Learnability for Adversarial Noisy Bandits

arXiv:2605. 09200v2 Announce Type: replace Abstract: We study adversarial noisy bandits given a known function class $\mathcal{F}$.

By Steve Hanneke, Kun Wang

reinforcement-learning safety

arXiv AI

Jul 1

HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation

arXiv:2606. 30966v1 Announce Type: new Abstract: Formal specification is a powerful tool to guide the learning process and provides significant advantages over reward shaping: (1) mathematical rigor; (2) expressiveness to specify objectives and constraints, and (3) the ability to define tactics to achieve objectives.

By Arshia Rafieioskouei, Tzu-Han Hsu, Matthew Lucas, Borzoo Bonakdarpour

agents reinforcement-learning benchmarks

arXiv AI

Jul 1

Corruption Robust Offline Reinforcement Learning with Human Feedback

arXiv:2402. 06734v2 Announce Type: replace-cross Abstract: We study data corruption robustness for reinforcement learning with human feedback (RLHF) in an offline setting.

By Debmalya Mandal, Andi Nika, Parameswaran Kamalaruban, Adish Singla, Goran Radanovi\'c

reinforcement-learning safety

arXiv AI

Jul 1

Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models

arXiv:2606. 31846v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models offer a promising framework for robotic manipulation by connecting language instructions, visual observations, and continuous control.

By Lang Cao, Renhong Chen, Luyi Li, Peng Wang, Mofan Peng, Yitong Li

reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv AI

Jul 1

LLM-Empowered Agentic MAC Protocols: A Dynamic Stackelberg Game Approach

arXiv:2510. 10895v2 Announce Type: replace Abstract: Medium Access Control (MAC) protocols, essential for wireless networks, are typically manually configured.

By Renxuan Tan, Rongpeng Li, Fei Wang, Chenghui Peng, Shaoyun Wu, Zhifeng Zhao, Honggang Zhang

llms agents reinforcement-learning efficiency safety

arXiv Machine Learning

Jul 1

Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions

arXiv:2606. 31769v1 Announce Type: new Abstract: We study policy optimization for online episodic tabular Markov decision processes with unknown transition kernels, aiming for best-of-both-worlds guarantees together with data-dependent regret bounds.

By Mingyi Li, Taira Tsuchiya, Kenji Yamanishi

reinforcement-learning safety

arXiv AI

Jul 1

Xiaomi-GUI-0 Technical Report

arXiv:2606. 31410v1 Announce Type: new Abstract: Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, text entry, and navigation.

By Wanxia Cao, Chengzhen Duan, Pei Fu, Pengzhi Gao, Niu Lian, Fazhan Liu, Hui Liu, Heng Qu, Qinzhuo Wu, Zhehao Yu, Tongbo Chen, Shiqi Cui, Anan Du, Shukai Jia, Yuanfa Li, Yike Liu, Wenchao Lu, Haoyuan Sun, Jiatong Sun, Cheng Tan, Yajie Wang, Changqiao Wu, Tao Xiong, Jiahui Yang, Yuxuan Yuan, Ruoceng Zhang, Shaojie Zhang, Jian Zhu, Jian Luan, Cong Zou

llms agents reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Jul 1

Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity

arXiv:2602. 03778v2 Announce Type: replace-cross Abstract: Tail-end risk measures such as static conditional value-at-risk (CVaR) are used in safety-critical applications to prevent rare, yet catastrophic events.

By Aneri Muni, Vincent Taboga, Esther Derman, Pierre-Luc Bacon, Erick Delage

reinforcement-learning safety

Get the daily briefing