Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,242 stories · RSS feed

arXiv AI

Jun 30

Inference-Time Diversity in RL-Trained Lean Theorem Provers: A Diagnostic Study

arXiv:2601. 16172v3 Announce Type: replace Abstract: RL-trained Lean theorem provers mode-collapse at inference time: on miniF2F-test with DeepSeek-Prover-V1.

By Zachary Burton

reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 30

Dockerless: Environment-Free Program Verifier for Coding Agents

arXiv:2606. 28436v1 Announce Type: cross Abstract: Program verifiers play a central role in training coding agents, including selecting trajectories for supervised fine-tuning (SFT) and providing rewards for reinforcement learning (RL).

By Wenhao Zeng, Yuling Shi, Xiaodong Gu, Chao Hu, Chaofan Wang, Yuhao Cui, Hongting Zhou, Mengnan Qi, Jianqiao Wangni, Zhaojian Yu, Shuzheng Gao, Kai Cai, Shilin He

agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Jun 30

Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation

arXiv:2606. 28998v1 Announce Type: cross Abstract: Large Language Model (LLM) alignment trains an LLM using preference data to produce outputs that better meet established quality standards.

By Gias Uddin, Sanjeepan Sivapiran

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jun 30

Trust Your Instincts: Confidence-Driven Test-Time RL for Vision-Language-Action Models

arXiv:2606. 29892v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become indispensable for pushing Vision-Language-Action Models (VLAs) beyond static imitation learning.

By Siyao Chen, Jiakang Yuan, Jiaxin Wang, Tao Chen

reinforcement-learning multimodal benchmarks

arXiv AI

Jun 30

Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models

arXiv:2606. 30627v1 Announce Type: cross Abstract: Conservative offline training is widely advocated as a safe foundation for subsequent online adaptation: if a policy stays close to well-supported behaviour, the argument goes, it is less likely to exploit imperfections in a learned reward model.

By Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary

reinforcement-learning safety

arXiv Machine Learning

Jun 30

Hierarchical Decision Making with Structured Policies: A Principled Design via Inverse Optimization

arXiv:2606. 28764v1 Announce Type: new Abstract: Hierarchical decision-making frameworks are pivotal for addressing complex control tasks, enabling agents to decompose intricate problems into manageable subgoals.

By Yuexuan Wang, Jingyuan Zhou, Kaidi Yang

agents reinforcement-learning

arXiv AI

Jun 30

Reinforcement Learning for Software Vulnerability Analysis: A Systematic Review with Emphasis on C/C++ Source Code and Static Analysis

arXiv:2606. 28403v1 Announce Type: cross Abstract: Vulnerability detection in C/C++ software remains a major security challenge due to code complexity, manual memory management, and the limitations of traditional static analysis.

By Bruno Caro-V\'asquez, Carola Figueroa-Flores, Gast\'on Marquez

agents reinforcement-learning benchmarks

arXiv AI

Jun 30

AutoB2G: Agentic Simulation and Reinforcement Learning for Spatio-Temporal Grid-Interactive Building Control

arXiv:2603. 26005v2 Announce Type: replace Abstract: Grid-interactive building control has emerged as a promising approach for improving demand-side flexibility in modern power systems.

By Borui Zhang, Nariman Mahdavi, Subbu Sethuvenkatraman, Flora Salim

llms agents reinforcement-learning

arXiv AI

Jun 30

Be Faithful When Response: Returning Fluent and Grounded Answers for Vision-Language Models Reinforcement Learning

arXiv:2606. 29984v1 Announce Type: new Abstract: Reinforcement Learning (RL) is an important paradigm for improving the reasoning capabilities of Vision-Language Models (VLMs).

By Peng, Lee, Yin Zhang, Yanglin Zhang, Haonan Wu, Zishan Liu, Ruoxi Zang, Xin Zhu, Jiayin Zheng, Jian Yao, Zefeng Ji, Fei Ma

llms reinforcement-learning multimodal benchmarks

arXiv AI

Jun 30

TERC: A Transfer Entropy Redundancy Criterion for State Variable Selection in Reinforcement Learning

arXiv:2401. 11512v2 Announce Type: replace-cross Abstract: Identifying the most suitable variables to represent the state is a fundamental challenge in Reinforcement Learning (RL).

By Charles Westphal, Stephen Hailes, Mirco Musolesi

agents reinforcement-learning benchmarks

arXiv AI

Jun 30

Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization

arXiv:2602. 11351v2 Announce Type: replace Abstract: Proactive large language model (LLM) agents aim to actively plan, query, and interact over multiple turns, enabling efficient task completion beyond passive instruction following and making them essential for real-world, user-centric applications.

By Yihang Yao, Zhepeng Cen, Haohong Lin, Shiqi Liu, Zuxin Liu, Jiacheng Zhu, Zhang-Wei Hong, Laixi Shi, Ding Zhao

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 30

AI Trading's Alpha Singularity: Emergent Market Reasoning through Agent-to-Agent Self-Evolution

arXiv:2606. 29194v1 Announce Type: new Abstract: Automated alpha mining holds the scoring function fixed and varies the search algorithm over it.

By Yuqi Li, Siyuan Liu, Bingjun Liu

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 30

The Two Genie Game: Adoption and Welfare in Audit-Grounded AI Governance

arXiv:2606. 28710v1 Announce Type: new Abstract: We ask under what conditions an agent with a harm-minimizing policy can displace an approval-seeking (RLHF) agent in a competitive market, and when that policy is sufficient to prevent community harm.

By Darrell Lewis-Sandy

agents reinforcement-learning safety

arXiv Machine Learning

Jun 30

FlowAWR: Online Adaptive Flow Reinforcement via Advantage-Weighted Rectification

arXiv:2606. 30376v1 Announce Type: new Abstract: Aligning generative flow models on continuous spaces via online reinforcement learning is constrained by intractable trajectory likelihoods.

By Zheming Fu, Ruizhe He, Wei Shang, Xiaoxiao Ma, Lei Wang, Chang Liu, Siming Fu

reinforcement-learning safety

arXiv Machine Learning

Jun 30

Constrained Policy Optimization with Cantelli-Bounded Value-at-Risk

arXiv:2601. 22993v4 Announce Type: replace Abstract: We introduce Canary, a risk-averse method designed to optimize Value-at-Risk (VaR) constrained reinforcement learning (RL) problems.

By Rohan Tangri, Jan-Peter Calliess

reinforcement-learning benchmarks safety

arXiv Machine Learning

Jun 30

Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation

arXiv:2602. 09305v2 Announce Type: replace Abstract: Large Language Models (LLMs) demonstrate transformative potential, yet their reasoning remains inconsistent and unreliable.

By Pei-Chi Pan, Yingbin Liang, Sen Lin

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jun 30

Synthetic Interaction Data for Scalable Personalization in Large Language Models

arXiv:2602. 12394v2 Announce Type: replace Abstract: Personalized prompting offers large opportunities for deploying large language models (LLMs) to diverse users, yet existing prompt optimization methods primarily focus on task-level optimization while largely overlooking user-specific preferences and latent constraints of individual users.

By Yuchen Ma, Yue Huang, Wenjie Wang, Xiaonan Luo, Xiangliang Zhang, Stefan Feuerriegel

llms agents reinforcement-learning benchmarks safety

arXiv AI

Jun 30

Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verification

arXiv:2606. 29023v1 Announce Type: cross Abstract: Spatio-temporal grounding in long videos requires precise temporal localization and robust object tracking conditioned on natural-language queries.

By Tianshu Zhang, Yan Wang, Ji Qi, Lijie Wen

llms reinforcement-learning multimodal

arXiv AI

Jun 30

Domain Adaptation with Adaptive Imagination for Visual Reinforcement Learning under Limited Target Data

arXiv:2606. 30192v1 Announce Type: new Abstract: Sim-to-real transfer remains a major obstacle for reinforcement learning (RL), especially for vision-based control where image observations exacerbate the state-distribution shift between simulation and the real world.

By Hyunwoo Park, Sang-Hyun Lee

reinforcement-learning robotics fine-tuning

arXiv AI

Jun 30

Exploration and Online Transfer with Behavioral Foundation Models

arXiv:2606. 29980v1 Announce Type: new Abstract: Zero-shot Transfer in Reinforcement Learning (RL) aims to train an agent that can generate optimal policies for any reward function, without additional learning at transfer time, while training only on reward-free trajectories.

By Louis Bagot (SyCoSMA), Mathieu Lefort (LIRIS, SyCoSMA, IRISA, MALT, UR), La\"etitia Matignon (SyCoSMA)

agents reinforcement-learning

Get the daily briefing