Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,740 stories · RSS feed

arXiv AI

Sep 30

UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training

arXiv:2609.38043v1 Announce Type: new Abstract: Interactive agent benchmarks and multi-turn reinforcement learning increasingly place a second language model in the role of the user. This simulated u...

By Ashish Jain, Armaan Sandhu

llms agents reinforcement-learning benchmarks

arXiv AI

Sep 30

AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation

arXiv:2609.38142v1 Announce Type: new Abstract: A small trainable advisor can steer a frozen language-model executor using natural-language advice. In addition to learning from task rewards, the advi...

By Rishabh Agrawal, Hejie Cui, Shasha Li, Shanchan Wu, Sercan \"{O}. Ar{\i}k

llms reinforcement-learning efficiency

arXiv AI

Sep 30

Targeting Pivotal Decisions for Credit Assignment in Agentic Reinforcement Learning

arXiv:2609.36178v1 Announce Type: cross Abstract: Group Relative Policy Optimization (GRPO) has become a promising approach for training large language model agents. However, its uniform assignment o...

By Dongwon Jung, Hemanth Neelgund Ramesh, Yifan Wang, Xiaomin Li, Yuexing Hao, Yu Hu, Muhao Chen, Varun Chandrasekaran, Andrzej Banburski-Fahey, Jaron Lanier

llms agents reinforcement-learning

arXiv AI

Sep 30

Visual Parallel Search: Learning to Search High-Resolution Images with Parallel Tile Inspection and Adaptive Zoom

arXiv:2609.37002v1 Announce Type: cross Abstract: High-resolution visual question answering often fails because a multimodal model does not acquire the small, spatially localized evidence needed to a...

By Xijia Tao, Yihua Teng, Xinyu Fu, Cheng Gong, Ziru Liu, Xudong Xie, Rui Liu, Lingpeng Kong

agents nlp reinforcement-learning fine-tuning multimodal benchmarks

arXiv AI

Sep 30

What Does Post-Training Change in Multilingual Reasoning?

arXiv:2609.37104v1 Announce Type: cross Abstract: Open-source reasoning models provide unequal access to reasoning capability across languages. When a model can solve a problem but cannot deliver a c...

By Hongyang Li, Xiao Li, Caesar Wu, Gr\'egoire Danoy, Pascal Bouvry

reinforcement-learning fine-tuning

arXiv AI

Sep 30

PowerMarketJax: A JAX Benchmark Suite for Multi-Agent Reinforcement Learning in Power Markets

arXiv:2609.37321v1 Announce Type: cross Abstract: Power markets are a natural testbed for multi-agent reinforcement learning (MARL), where multiple self-interested participants repeatedly submit bids...

By Zhanhua Pan, Xin Qin, Xiao Liu, Zhilong Cao, Jianhong Wang, Dawei Qiu

agents reinforcement-learning benchmarks

arXiv AI

Sep 30

Spatial Memory Agent: Experience-Grounded Procedural Memory for Spatial Intelligence

arXiv:2608.12743v3 Announce Type: replace Abstract: Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ab...

By Haokai Zhang, Yuhang Ding, Yunshu Zhou, Xinze Du, Shengtao Zhang, Zhiyue Zhao, Yuling Xi, Hao Chen

agents reinforcement-learning robotics fine-tuning multimodal benchmarks

arXiv AI

Sep 30

Asymptotic Universal Alignment: A New Alignment Framework via Test-Time Scaling

arXiv:2601.08777v2 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) to serve users with heterogeneous and potentially conflicting preferences is a central challenge for pe...

By Yang Cai, Weiqiang Zheng

llms reinforcement-learning safety

arXiv AI

Sep 30

Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Study

arXiv:2608.00042v2 Announce Type: replace-cross Abstract: Domain adaptation of small language models (SLMs) has emerged as a practical strategy for deploying capable NLP systems in resource-constrain...

By Ramesh B. Paramkusham

llms nlp reinforcement-learning fine-tuning multimodal benchmarks safety

arXiv AI

Sep 30

Diffusion Reward Models

arXiv:2609.33803v2 Announce Type: replace-cross Abstract: Reward models underpin the alignment of large language models, yet the dominant designs reduce each prompt--response pair to a point estimate...

By Xiangyang Wang, Bingxiang He, Zeyuan Liu, Jiaze Wang, Ziqing Qiao, Yuxin Zuo, Huan-ang Gao, Cheng Qian, Wenbin Zhang, Ran Li, Youbang Sun, Ning Ding, Yuanchun Shi, Zhiyuan Liu, Chaojun Xiao, Chun Yu

llms diffusion reinforcement-learning multimodal benchmarks safety

arXiv Machine Learning

Sep 30

Understanding LLM Parameter Update Sparsity through the Lens of Fisher

arXiv:2609.36262v1 Announce Type: new Abstract: Recent studies have observed that parameter changes during language-model post-training can be concentrated in a small subset of coordinates. This phen...

By Yufan Zhang, Sagnik Mukherjee, Hao Peng

llms reinforcement-learning fine-tuning efficiency

arXiv Machine Learning

Sep 30

Optimal Multi-Reward Reinforcement Learning

arXiv:2609.36486v1 Announce Type: new Abstract: We study an unknown-transition finite-horizon Markov decision process (MDP) with a finite collection of known reward functions $\{r^1, r^2, \ldots, r^M...

By Zijun Chen, Zihan Zhang

reinforcement-learning

arXiv Machine Learning

Sep 30

EasyPPO: Stabilizing the Critic Is Key

arXiv:2609.36802v1 Announce Type: new Abstract: A key strength of Proximal Policy Optimization (PPO) is its learned critic, which uses historical trajectories collected during reinforcement learning...

By Xuanyi Zhou, Qiuyang Mang, Huanzhi Mao, Dacheng Li, Wenhao Chai, Mayank Mishra, Yichuan Wang, Karthik Narasimhan, Alvin Cheung, Joseph E. Gonzalez

llms reinforcement-learning

arXiv Machine Learning

Sep 30

Where the Model Changes Its Mind: Hindsight-Divergence Localization for Efficient Reinforcement Learning with Verifiable Rewards

arXiv:2609.36864v1 Announce Type: new Abstract: Group-relative methods for reinforcement learning with verifiable rewards (RLVR) learn from differences in rollout outcomes. Independently sampling com...

By Fanchao Chen, Hengyu Fu, Shivaram Venkataraman, Jiantao Jiao

agents reinforcement-learning

arXiv Machine Learning

Sep 30

Sample Complexity of Equivariant Reinforcement Learning

arXiv:2609.36421v1 Announce Type: cross Abstract: Reinforcement learning (RL) is a powerful framework for robotic control, yet its practical application is often hindered by high sample complexity. T...

By Rayan Mazouz, Haibo Zhao, Chris Hillar, Christian Shewmake

reinforcement-learning robotics

arXiv Machine Learning

Sep 30

Stochastic Decision Horizons for Survival-Based Constrained Reinforcement Learning

arXiv:2602.04599v3 Announce Type: replace Abstract: We propose stochastic decision horizons (SDH), a theoretically grounded framework for survival-based constrained reinforcement learning from per-st...

By Nikola Milosevic, Leonard Franz, Daniel Haeufle, Georg Martius, Nico Scherf, Pavel Kolev

reinforcement-learning safety

arXiv AI

Sep 30

Engineering Efficient Self-Play Chess: Search, Replay, and Throughput Under Limited Compute

arXiv:2609.37447v1 Announce Type: cross Abstract: How strong can an AlphaZero-style chess system become under limited training compute when its entire learning loop is engineered for efficiency? We t...

By Bertil Braun

reinforcement-learning efficiency benchmarks

arXiv AI

Sep 30

MLToolBench: Learning Tool-Augmented Agents for Machine Learning Development

arXiv:2609.36679v1 Announce Type: new Abstract: Machine learning engineering (MLE) agents have made substantial progress, but learning through ML experimentation remains costly in time and computatio...

By Xin Yu, Lizhu Zhang, Jiamu Bai, Yanhong Wu, Zellux Wang, Serena Li, Weiwei Li, Lingzhou Xue, Xiangjun Fan, Bo Peng

agents reinforcement-learning fine-tuning

arXiv Computation and Language

Sep 30

Question-Specific Knowledge Graphs for Efficient Visual Reasoning

arXiv:2609.35942v1 Announce Type: new Abstract: Recent work in visual question answering has shown that vision-language models can exhibit strong reasoning capabilities by translating visual inputs i...

By Ting-Chih Chen, Emile van Krieken, Shujian Yu, Filip Ilievski

llms nlp reinforcement-learning multimodal benchmarks

arXiv Computation and Language

Sep 30

Momentum-Coupled Rubric Adaptation for Detailed Image Captioning

arXiv:2609.36893v1 Announce Type: new Abstract: Detailed image captioning requires accurate and comprehensive descriptions of fine-grained visual content, yet caption quality spans factual accuracy,...

By Zhenwen Ji, Lei Jin, Shanyong Wang, Jiaming Lu, Chengqiang Lu, Yi Wu, Yao Hu, Lizhen Cui, Yanyu Xu

llms nlp reinforcement-learning fine-tuning benchmarks

Get the daily briefing