Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

3,470 stories · RSS feed

arXiv AI

Aug 11

Understanding and Mitigating Premature Confidence for Better LLM Reasoning

arXiv:2605. 24396v2 Announce Type: replace Abstract: Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps, limiting the gains from additional test-time compute.

By Jingchu Gai, Guanning Zeng, Christina Baek, Chen Wu, J. Zico Kolter, Andrej Risteski, Aditi Raghunathan

llms reinforcement-learning benchmarks safety

arXiv AI

Aug 11

FinTrace: Holistic Trajectory-Level Evaluation of LLM Tool Calling for Long-Horizon Financial Tasks

arXiv:2604. 10015v3 Announce Type: replace Abstract: Recent studies demonstrate that tool-calling capability enables large language models (LLMs) to interact with external environments for long-horizon financial tasks.

By Yupeng Cao, Haohang Li, Weijin Liu, Wenbo Cao, Anke Xu, Lingfei Qian, Xueqing Peng, Minxue Tang, Zhiyuan Yao, Jimin Huang, K. P. Subbalakshmi, Zining Zhu, Jordan W. Suchow, Yangyang Yu

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Aug 11

RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning

arXiv:2608. 09123v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) for open-ended tasks is challenging because responses must satisfy multidimensional criteria without following a single correct generation trajectory.

By Jinkun Hou, Zhuo Liu, Huimin Ren, Hongsheng Xin, Pan Zhou, Kun Zhan

llms reinforcement-learning benchmarks

arXiv AI

Aug 11

RecoverFly: A Failure-Aware Reinforcement Learning Post-Training Framework for Aerial Vision-Language Navigation

arXiv:2608. 09467v1 Announce Type: cross Abstract: Unmanned aerial vehicle vision-language navigation (UAV-VLN) requires agents to translate visual observations and language instructions into reliable flight actions in complex environments.

By Boxiong Wang, Hui Kang, Geng Sun, Jiahui Li, Chao Yu, Daxin Tian

agents reinforcement-learning multimodal benchmarks

arXiv AI

Aug 11

Collaborative Multi-Agent Scripts Generation for Enhancing Imperfect-Information Reasoning in Murder Mystery Games

arXiv:2604. 11741v2 Announce Type: replace Abstract: Vision-language models (VLMs) have shown impressive capabilities in perceptual tasks, yet they degrade in complex multi-hop reasoning under multiplayer game settings with imperfect and deceptive information.

By Keyang Zhong, Junlin Xie, Hefeng Wu, Haofeng Li, Guanbin Li

llms agents reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Aug 11

SuperCoder: Assembly Program Superoptimization with Large Language Models

arXiv:2505. 11480v4 Announce Type: replace-cross Abstract: Superoptimization is the task of transforming a program into a faster one, and ideally the very fastest possible one, while preserving its input-output behavior.

By Anjiang Wei, Tarun Suresh, Huanmi Tan, Yinglun Xu, Gagandeep Singh, Ke Wang, Alex Aiken

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Aug 11

CODS: Iterative Bellman-Residual Data Selection for Reusable Offline Reinforcement Learning

arXiv:2608. 07719v1 Announce Type: new Abstract: Offline reinforcement learning repeatedly trains policies from a fixed transition pool, making redundant data costly across seeds and hyperparameters, while naive subsampling can remove rare transitions needed for long-horizon credit assignment.

By Ibne Farabi Shihab, Sanjeda Akter, Abu Sa-Adat Mohamed Moon-Im Al Ahsan, Md Najmus Swaqeeb, Anuj Sharma

reinforcement-learning

arXiv AI

Aug 11

Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models

arXiv:2608. 09435v1 Announce Type: new Abstract: Understanding dynamic sound sources requires jointly determining what produces a sound, where the source is located, and how it moves over time.

By Zhi Zeng, Cheng Zhang, Zesheng Yang, Rendong Pi, Jiaying Wu, Di Zhang, Zihan Ma, Guodong Li, Zhou Yang, Yu Xiang, Yifei Zheng, Minnan Luo

llms reinforcement-learning multimodal benchmarks

arXiv AI

Aug 11

SAKE: Structured Agentic Knowledge Extrapolation for Complex LLM Reasoning via Reinforcement Learning

arXiv:2505. 15062v5 Announce Type: replace-cross Abstract: Knowledge extrapolation is the process of inferring novel information by combining and extending existing knowledge that is explicitly available.

By Jiashu He, Jinxuan Fan, Bowen Jiang, Ignacio Houine, Dan Roth, Alejandro Ribeiro

llms agents reinforcement-learning fine-tuning benchmarks

arXiv AI

Aug 11

LLM Reasoning for Subjective Tasks: Failure Modes, Mitigation, and Dynamic Reasoning Routing

arXiv:2608. 08889v1 Announce Type: new Abstract: Recommendation systems thrive on personalization, where ''correctness'' is rarely a binary truth but a matter of subjective human preference.

By Juncheng Dong, Ding Tong, Ishan Gupta, Yuyan Wang

llms agents reinforcement-learning safety

arXiv Machine Learning

Aug 11

Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring

arXiv:2605. 00754v4 Announce Type: replace-cross Abstract: Reward models (RMs) have become an indispensable fixture of the language model (LM) post-training playbook, enabling policy alignment and test-time scaling.

By Indraneil Paul, Goran Glava\v{s}, Iryna Gurevych

llms reinforcement-learning benchmarks safety

arXiv AI

Aug 11

Improving Generalization Robustness of Multimodal RLVR

arXiv:2608. 08802v1 Announce Type: new Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) makes Multimodal Large Language Models more accurate, but the gains are brittle: simply paraphrasing a question or changing the prompt template can degrade them, which challenges reliable deployment in high-stakes scenarios like medical VQA.

By Pengfei Zhou, Zhiwei Tang, Xiaopeng Peng, Chenrui Zhou, Lama Moukheiber, Yixing Ma, Bin Xu, Jiajun Song, Zhenglin Wan, Wangbo Zhao, Jiasheng Tang, Bohan Zhuang, Fan Wang, Yang You

llms rag reinforcement-learning multimodal benchmarks

arXiv AI

Aug 11

Critic-Free Deep Reinforcement Learning for Maritime Coverage Path Planning on Irregular Hexagonal Grids

arXiv:2603. 28385v2 Announce Type: replace-cross Abstract: Maritime surveillance missions, such as search and rescue and environmental monitoring, rely on the efficient allocation of sensing assets over vast and geometrically complex areas.

By Carlos S. Sep\'ulveda, Gonzalo A. Ruz

llms reinforcement-learning

arXiv AI

Aug 11

DocAtlas: Long-Document Understanding as Mutable-State Interaction

arXiv:2608. 07527v1 Announce Type: cross Abstract: Long-document understanding requires models to find and combine evidence across many pages, layouts, tables, figures, and charts.

By Hongchen Wei, Yuanzhe Wang, Bei Liu, Yifan Yang, Qi Dai, Kai Qiu, Yunsheng Li, Dongdong Chen, Chong Luo, Zhenzhong Chen, Baining Guo

llms rag agents reinforcement-learning multimodal

arXiv AI

Aug 11

Reflex First, Reflect Later: Latency-Aware Embodied LLM Agents for Dynamic Response

arXiv:2506. 07223v2 Announce Type: replace Abstract: Large language models (LLMs) have substantially improved the planning capabilities of embodied agents, enabling their deployment in dynamic and safety-critical environments.

By Yangqing Zheng, Shunqi Mao, Dingxin Zhang, Weidong Cai

llms agents reinforcement-learning robotics efficiency safety

arXiv AI

Aug 11

DUET: A Diversity-Quality Duet of Distillation Experts for Two-Step Video Generation

arXiv:2608. 09637v1 Announce Type: cross Abstract: Diffusion models have enabled high-quality video generation in recent years, but the high cost of iterative sampling hinders their practical deployment.

By Zian Li, Litong Gong, Borui Liao, Pengfei Liu, Xinyu Wang, Xinyuan Wei, Yifan Gao, Tiezheng Ge, Muhan Zhang

diffusion reinforcement-learning efficiency

arXiv AI

Aug 11

Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewards

arXiv:2608. 07531v1 Announce Type: cross Abstract: Search-augmented language agents should retrieve external information only when necessary and ground their answers in retrieved evidence.

By Cheng Ruoxi, Ma Haoxuan, Zhang Hongyi, Zhang Junming, Duan Ranjie, Xia Qiaolin, Wang Hao, Lu Yu, Shi Haibo, Ma Xingjun

llms agents reinforcement-learning benchmarks

arXiv AI

Aug 11

TrustRoboReward: Preference-Ordered Isotonic Score Editing for Multi-Paradigm Robot Reward Models

arXiv:2608. 08491v1 Announce Type: new Abstract: Reward models are a bottleneck for reinforcement learning in embodied AI.

By Yidong Wang, Yan Zhan, Ziteng Feng, Zhenyu Cui, Ziyi Zhou, Renzhao Liang, Jiaxuan Zhu, Zilei Yang, Yiran Zhao, Zhongkuan Mao, Bo Jia, Hanchu Ni, Chenggang Xie, Biao Liu, Yi Zhang, Yong Dai, Xiaozhu Ju, Wei Ye, Shikun Zhang

llms reinforcement-learning robotics multimodal benchmarks

arXiv AI

Aug 11

DistillCache: KL-Guided Adaptive KV-Cache Eviction for Memory-Efficient LLM Inference

arXiv:2608. 08878v1 Announce Type: cross Abstract: Transformer-based large language models (LLMs) achieve strong performance across many tasks, but their Key-Value (KV) cache grows linearly with sequence length, creating a severe memory bottleneck for long-context inference.

By Asaad Althoubi

llms reinforcement-learning efficiency benchmarks

arXiv AI

Aug 11

Goedel-Code-Prover: Hierarchical Proof Search for Open State-of-the-Art Code Verification

arXiv:2603. 19329v3 Announce Type: replace-cross Abstract: Large language models (LLMs) can generate plausible code but offer limited guarantees of correctness.

By Zenan Li, Ziran Yang, Deyuan He, Haoyu Zhao, Andrew Zhao, Shange Tang, Kaiyu Yang, Aarti Gupta, Zhendong Su, Chi Jin

llms reinforcement-learning benchmarks