Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

3,470 stories · RSS feed

arXiv AI

Aug 13

Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injection

arXiv:2608. 11977v1 Announce Type: new Abstract: Tool-using LLM agents are commonly trained and evaluated in environments where tool calls succeed reliably, yet deployed tools can fail transiently, persistently, or silently.

By Chaoran Chen, Vy Nguyen, Ziji Zhang, Abhinav Gullapalli, Ziyi Wang, Yuxuan Lu, Dakuo Wang, Jing Huang, Zhou Yu, Jin Lai

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Aug 13

LODESTAR: Trustworthy Entropy Is Navigated, Not Merely Measured -- Reinforced Polarizer Keeps a Frozen LLM from Being Confidently Misled by the Wrong Evidence

arXiv:2608. 11922v1 Announce Type: cross Abstract: Predictive-distribution entropy makes a strong selection rule in retrieval-augmented question answering: across five QA benchmarks, keeping the candidate answer that a frozen respondent LLM produces with the lowest answer-token entropy lifts mean answer $F_1$ from 0.

By Po-Jen Ko, Che-Cheng Wu, Hung-Chun Hsu, Li-Yang Chang, Chuan-Ju Wang

llms rag nlp reinforcement-learning benchmarks

arXiv AI

Aug 13

Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoning

arXiv:2608. 11260v1 Announce Type: new Abstract: Video Anomaly Detection (VAD) aims to identify anomalous events and localize their temporal intervals.

By Shibo Gao, Peipei Yang, Xu-Yao Zhang, Linlin Huang

llms agents reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv Machine Learning

Aug 13

Language-Structured Relational Q-Learning for Threat-Aware Control in Safety-Critical Driving

arXiv:2608. 11498v1 Announce Type: cross Abstract: Natural-language-based scenario generation offers an intuitive means of describing rare and complex driving interactions, yet it is still uncertain whether training with language-structured data leads to truly adaptive control policies.

By Aditya Humnabadkar, Huaizhong Zhang, Ardhendu Behera

reinforcement-learning safety

arXiv AI

Aug 13

From Numbers to Judgment: Specialist LLM Agents and Reinforcement Learning for European Listed Real Estate

arXiv:2608. 11381v1 Announce Type: new Abstract: We study whether the localized numerical operations and integrative judgments of financial analysis benefit from the same form of LLM specialization.

By Pardis Taghavi, Santosh Bhavani

llms agents reinforcement-learning

arXiv Machine Learning

Aug 13

Diffusion-Guided Cooperative Policy Learning for Target Tracking Based on Underwater Mobile Agent Networks

arXiv:2603. 29426v2 Announce Type: replace-cross Abstract: Multi-agent reinforcement learning (MARL) provides a promising solution for cooperative target tracking in networks of autonomous underwater vehicles (AUVs).

By Jiaao Ma, Chuan Lin, Guangjie Han, Shengchao Zhu, Zhenyu Wang, Chen An

agents diffusion reinforcement-learning

arXiv Machine Learning

Aug 13

Unmasking Toxic Mimicry in Medical Offline Reinforcement Learning for ICU Sepsis Management via Counterfactual Clinical Audits

arXiv:2608. 11410v1 Announce Type: new Abstract: Offline reinforcement learning (RL) offers considerable promise for optimizing ICU treatment decisions, yet standard evaluation metrics Mean Squared Error (MSE) and Fitted Q-Evaluation (FQE) assess only behavioral imitation and cannot detect Toxic Mimicry, a failure mode in which agents replicate harmful patterns such as treatment withdrawal during comfort-care transitions.

By Hangqi Ren, Junyi Liao

llms agents reinforcement-learning safety

arXiv Machine Learning

Aug 13

Mind the Gap: Structure-Aware Consistency in Preference Learning

arXiv:2604. 27733v2 Announce Type: replace Abstract: Aligning Large Language Models (LLMs) with human intent, whether through explicit reward modeling or direct methods such as DPO, fundamentally relies on minimizing a surrogate loss as a proxy for the true pairwise ranking objective.

By Mehryar Mohri, Yutao Zhong

llms reinforcement-learning safety

arXiv AI

Aug 13

One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL

arXiv:2608. 12253v1 Announce Type: cross Abstract: Multi-agent reinforcement learning for human-AI interaction typically relies on a single large language model to simulate user behavior.

By Simon Yu, Nicholas Tomlin, Marwa Abdulhai, Ximing Lu, Derek Chong, Abe Hou, Dilara Soylu, Sergey Levine, Christopher D. Manning, Weiyan Shi

llms agents reinforcement-learning benchmarks

arXiv AI

Aug 13

GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs

arXiv:2608. 11674v1 Announce Type: cross Abstract: On-policy rollout methods such as GRPO are central to post-training of large language models, yet they frequently suffer from training instabilities, cross-task capability degradation, and response-length inflation.

By Kai Yang, Jingwei Xu, Wanyu Wang, Kai-Yuan Guo, Zhenbo Yu, Yi Wang, Yu Qiao

llms reinforcement-learning

arXiv AI

Aug 13

Hamilton-Zero: A Neural Tensor-Network Foundation Model for Ground States of Arbitrary Quadratic Qubit Hamiltonians

arXiv:2608. 11911v1 Announce Type: cross Abstract: A central promise of useful quantum advantage is the ability to compute ground states of Hamiltonian systems beyond the reach of classical simulation methods.

By Timothy Heightman, Elena Orlova, Philip Mantrov, Aleksei Ustimenko

llms reinforcement-learning fine-tuning

arXiv AI

Aug 13

CLAIM: Leading Open-domain Active Clarification of Large Language Models with Uncertainty Measurement

arXiv:2608. 11631v1 Announce Type: new Abstract: In open-domain human-computer interaction scenarios, large language models (LLMs) frequently encounter user queries that are ambiguous or incomplete.

By Kuangzhao Yang, Ziliang Zhao, Zhicheng Dou

llms reinforcement-learning fine-tuning safety

arXiv AI

Aug 13

Backtrader-Bench: Benchmarking LLM Agents on Algorithmic Trading with Self-Generated MCQs

arXiv:2608. 11232v1 Announce Type: cross Abstract: Evaluating LLM coding agents in algorithmic trading is difficult because static benchmarks risk data contamination and numerical backtest outputs require ground truth from actual code execution.

By Ruoxi Zhao, Maziar Raissi

llms agents reinforcement-learning benchmarks

arXiv AI

Aug 13

Machine Learning-Based Cyber Defense for Cloud Infrastructure: An Adaptive Deep Q-Network Architecture for Intelligent Intrusion Detection and Automated Threat Mitigation

arXiv:2608. 12190v1 Announce Type: cross Abstract: With the increasing complexity of cyber assaults in cloud environments, adaptable security solutions are needed that can support real-time detection and autonomous response.

By Md Yassir Mottalib, Md Yousuf, Eklachur Rahman Bhuiyan, S M Ahsan Habib, Sonjoy Kumar Dey, Md. Salahuddin Gazi, Molay Kumar Roy, Asaduzzaman Anik

agents reinforcement-learning

arXiv AI

Aug 13

DORA Explorer: Improving the Exploration Ability of LLMs Without Training

arXiv:2604. 17244v2 Announce Type: replace-cross Abstract: Large language model (LLM) agents for sequential decision-making struggle to produce diverse outputs.

By Priya Gurjar, Md Farhan Ishmam, Kenneth Marino

llms agents reinforcement-learning

arXiv AI

Aug 13

Benchmarking LLM Judges for Mobile Agent Evaluation

arXiv:2608. 11434v1 Announce Type: new Abstract: Mobile agent benchmarks increasingly rely on LLM-based judges to evaluate task completion, yet the reliability of these judges on mobile agent trajectories remains largely unexamined.

By Ziqiang Wan, Li Gu, Zhixiang Chi, Zhi Liu, Seyed Mehdi Ayyoubzadeh, Yuanhao Yu, Yang Wang

llms agents reinforcement-learning benchmarks

arXiv AI

Aug 13

TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning

arXiv:2605. 12236v2 Announce Type: replace-cross Abstract: Fine-tuning pre-trained robot policies with reinforcement learning (RL) often inherits the bottlenecks introduced by pre-training with behavioral cloning (BC), which produces narrow action distributions that lack the coverage necessary for downstream exploration.

By Matthew M. Hong, Jesse Zhang, Anusha Nagabandi, Abhishek Gupta

agents diffusion reinforcement-learning robotics fine-tuning

arXiv AI

Aug 13

Redistribution-based Cost Inference Improves Sparse Safe Offline RL

arXiv:2608. 12306v1 Announce Type: cross Abstract: Safe offline RL typically assumes access to dense per-step cost annotations, but in practice supervisors provide only trajectory-level stop-feedback: a binary signal at the first unsafe transition, with no per-step attribution.

By Ebenezer Gelo (University of the Witwatersrand), Geraud Nangue Tasse (University of the Witwatersrand), Steven James (University of the Witwatersrand), Benjamin Rosman (University of the Witwatersrand)

reinforcement-learning robotics

arXiv Machine Learning

Aug 13

PAIR: Pairwise-Aware Inclusion Reweighting for Adaptive Rollout Allocation in RLVR

arXiv:2608. 11368v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) spends most of its compute generating groups of long reasoning trajectories.

By Pixel Nomand, Elena Voss, Marcus Hale, Sofia Reyes

reinforcement-learning

arXiv AI

Aug 13

SCOUT: Unlocking Enhanced Spatial Reasoning via Structured Chain-of-Thought and Multi-Objective Process Reward

arXiv:2608. 12220v1 Announce Type: cross Abstract: Existing Vision-Language Models (VLMs) exhibits a critical bottleneck in robust spatial reasoning.

By Zile Zhou, Huining Yuan, Weichen Zhang, Xinlei Chen, Xiao-ping Zhang

llms reinforcement-learning multimodal benchmarks