Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,242 stories · RSS feed

arXiv AI

Jun 15

Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost

arXiv:2602. 03120v2 Announce Type: replace-cross Abstract: Post-Training Quantization (PTQ) is essential for deploying Large Language Models (LLMs) on memory-constrained devices, yet it renders models static and difficult to fine-tune.

By Yinggan Xu, Kajetan Schweighofer, Risto Miikkulainen, Xin Qiu

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 15

Diffusion Policy Optimization without Drifting Apart

arXiv:2606. 13795v1 Announce Type: new Abstract: RL post-training has become increasingly pivotal for improving diffusion policies, but existing diffusion policy-gradient methods are often unstable and cannot achieve reliable policy improvement.

By Haozhe Jiang, Haiwen Feng, Pieter Abbeel, Jiantao Jiao, Angjoo Kanazawa, Nika Haghtalab

llms diffusion reinforcement-learning efficiency

arXiv Machine Learning

Jun 15

Direct Preference Optimization for Chatbot Fine-Tuning: An Empirical Study

arXiv:2606. 12881v2 Announce Type: replace-cross Abstract: We present an approach to fine-tuning large language models using Direct Preference Optimization (DPO), a reinforcement learning technique.

By Dezhi Yu, Yvonne Qiu, ShuoJia Fu

llms reinforcement-learning fine-tuning

arXiv AI

Jun 15

ANSR-DT: A Neuro-Symbolic Framework for Adaptive and Explainable Digital Twins

arXiv:2501. 08561v4 Announce Type: replace Abstract: Digital twins are increasingly used to monitor and optimize industrial systems, yet many existing frameworks remain difficult to interpret, slow to adapt, and limited in their ability to incorporate explicit domain knowledge.

By Safayat Bin Hakim, Muhammad Adil, Alvaro Velasquez, Houbing Herbert Song

reinforcement-learning benchmarks safety

arXiv AI

Jun 15

X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs

arXiv:2603. 24596v3 Announce Type: replace-cross Abstract: While the shift from cascaded dialogue systems to end-to-end (E2E) speech Large Language Models (LLMs) improves latency and paralinguistic modeling, E2E models often exhibit a significant performance degradation compared to their text-based counterparts.

By Di Cao, Dongjie Fu, Hai Yu, Siqi Zheng, Xu Tan, Tao Jin

llms reinforcement-learning fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jun 15

Learning Coordinated Preference for Multi-Objective Multi-Agent Reinforcement Learning

arXiv:2606. 14693v1 Announce Type: cross Abstract: Cooperative multi-objective multi-agent reinforcement learning (MOMARL) models team decision making under multiple, potentially conflicting objectives.

By Pengxin Wang, Lihao Guo, Yi Xie, Bo Liu, Siyang Cao, Jingdi Chen

agents reinforcement-learning

arXiv Machine Learning

Jun 15

PERRY: Policy Evaluation with Confidence Intervals using Auxiliary Data

arXiv:2507. 20068v2 Announce Type: replace Abstract: Off-policy evaluation (OPE) methods estimate the value of a new reinforcement learning (RL) policy prior to deployment.

By Aishwarya Mandyam, Jason Meng, Ge Gao, Jiankai Sun, Mac Schwager, Barbara E. Engelhardt, Emma Brunskill

reinforcement-learning robotics

arXiv AI

Jun 15

OdysSim: Building Foundation Models for Human Behavior Simulation

arXiv:2606. 14199v1 Announce Type: cross Abstract: Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation.

By Xuhui Zhou, Weiwei Sun, Weihua Du, Jiarui Liu, Haojia Sun, Qianou Ma, Tongshuang Wu, Yiming Yang, Maarten Sap

llms reinforcement-learning efficiency benchmarks safety

arXiv AI

Jun 15

Deep Dense Exploration for LLM Reinforcement Learning via Pivot-Driven Resampling

arXiv:2602. 14169v2 Announce Type: replace-cross Abstract: Effective exploration is a key challenge in reinforcement learning for large language models: discovering high-quality trajectories within a limited sampling budget from the vast natural language sequence space.

By Yiran Guo, Zhongjian Qiao, Yingqi Xie, Jie Liu, Dan Ye, Ruiqing Zhang, Shuang Qiu, Lijie Xu

llms reinforcement-learning benchmarks safety

arXiv AI

Jun 15

Orchestra-o1: Omnimodal Agent Orchestration

arXiv:2606. 13707v1 Announce Type: new Abstract: The recent success of agent swarms has shifted the paradigm of large language model (LLM)-based agents from single-agent workflows to multi-agent systems, highlighting the importance of agent orchestration for task decomposition and collaboration.

By Fan Zhang, Vireo Zhang, Shengju Qian, Haoxuan Li, Hao Wu, Jinyang Wu, Donghao Zhou, Zhihong Zhu, Zheng Lian, Xin Wang, Pheng-Ann Heng

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 15

Safety-Contract Graph Multi-Agent Reinforcement Learning for Autonomous Network Security Response

arXiv:2606. 13832v1 Announce Type: cross Abstract: Autonomous network-security response systems promise to reduce Security Operations Centre (SOC) reaction latency, but reward-only multi-agent reinforcement learning (MARL) can improve security reward while remaining non-deployable.

By Jose Luis Lima de Jesus Silva

agents reinforcement-learning benchmarks safety

arXiv AI

Jun 15

Scalable Production Scheduling: Linear Complexity via Unified Homogeneous Graphs

arXiv:2604. 23841v2 Announce Type: replace-cross Abstract: Efficiently solving the Job Shop Scheduling Problem in real-world industrial applications requires policies that are both computationally lean and topologically robust.

By Jonathan Hoss, Moritz Link, Noah Klarmann

agents diffusion reinforcement-learning benchmarks

arXiv AI

Jun 15

From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI

arXiv:2606. 14502v1 Announce Type: new Abstract: Large Language Models (LLMs) are undergoing a fundamental transformation from conversational generators into integrated AI systems capable of reasoning, action, memory, and self-improvement.

By Yongheng Zhang, Ziang Liu, Jiaxuan Zhu, Shuai Wang, Xiangqi Chen, Haojing Huang, Jiayi Kuang, Siyu Chen, Ao Shen, Hao Wu, Qiufeng Wang, Qian-Wen Zhang, Junnan Dong, Wenhao Jiang, Ying Shen, Hai-Tao Zheng, Yinghui Li, Di Yin, Xing Sun, Philip S. Yu

llms agents reinforcement-learning benchmarks

arXiv AI

Jun 15

Tackling GNARLy Problems: Graph Neural Algorithmic Reasoning Reimagined through Reinforcement Learning

arXiv:2509. 18930v3 Announce Type: replace-cross Abstract: Neural algorithmic reasoning (NAR) is a paradigm that trains neural networks to execute classic algorithms by supervised learning.

By Alex Schutz, Victor-Alexandru Darvariu, Efimia Panagiotaki, Bruno Lacerda, Nick Hawes

reinforcement-learning

arXiv AI

Jun 15

Closing the Reflection Gap: A Free Calibration Bonus for Agentic RL

arXiv:2606. 14211v1 Announce Type: new Abstract: LLMs are increasingly deployed as agents that interact with external environments and observe feedback such as execution results, error messages, and tool outputs.

By Yinglun Zhu

llms agents reinforcement-learning benchmarks data-engineering

arXiv AI

Jun 15

Causal Object-Centric Models for Planning with Monte Carlo Tree Search

arXiv:2606. 14418v1 Announce Type: new Abstract: We introduce COMET (Causal Object-centric Model for Efficient Tree search), a model-based reinforcement learning algorithm that performs Monte Carlo Tree Search in a slot-structured latent space.

By Rodion Vakhitov, Leonid Ugadiarov, Alexey Skrynnik, Aleksandr Panov

llms diffusion reinforcement-learning benchmarks safety

arXiv AI

Jun 15

Planning with the Views via Scene Self-Exploration

arXiv:2605. 29563v2 Announce Type: replace Abstract: Can VLMs predict how each camera move changes the view, and plan many such moves ahead?

By Kangrui Wang, Linjie Li, Zhengyuan Yang, Shiqi Chen, Zihan Wang, Li Fei-Fei, Jiajun Wu, Leonidas Guibas, Lijuan Wang, Manling Li

llms reinforcement-learning efficiency

arXiv Machine Learning

Jun 15

DRIVE: Distributional and Retrieval-Augmented Bidding with Value Evaluation

arXiv:2606. 14192v1 Announce Type: new Abstract: Auto-bidding is a core component of real-time advertising systems, where decisions must optimize long-term performance under budget and cost constraints, while online exploration is prohibitively risky.

By Miduo Cui, Haochen Wang, Shangqin Mao, Xun Yang, Qianlong Xie, Xingxing Wang, Xuri Ge, Ying Zhou, Zhiwei Xu

llms rag reinforcement-learning benchmarks

arXiv AI

Jun 15

When Good Verifiers Go Bad: Self-Improving VLMs Can Regress on New Tasks

arXiv:2606. 14629v1 Announce Type: cross Abstract: Verifier-driven self-DPO is a common recipe for self-improving production visual-language models.

By Jianzhe Lin

llms reinforcement-learning

arXiv AI

Jun 15

3D-RFT: Reinforcement Fine-Tuning for Video-based 3D Scene Understanding

arXiv:2603. 04976v2 Announce Type: replace-cross Abstract: Reinforcement Learning with Verifiable Rewards ( RLVR ) has emerged as a transformative paradigm for enhancing the reasoning capabilities of Large Language Models ( LLMs), yet its potential in 3D scene understanding remains under-explored.

By Xiongkun Linghu, Jiangyong Huang, Baoxiong Jia, Siyuan Huang

llms reinforcement-learning fine-tuning multimodal benchmarks

Get the daily briefing