Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

3,495 stories · RSS feed

arXiv Machine Learning

Jul 31

HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks

arXiv:2607. 28301v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) can equip large language models (LLMs) with domain knowledge for high-performance computing (HPC) tasks such as data race detection and benchmark question answering.

By Tiangang Li, Xiangbo Tian

llms nlp reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 31

LM-GRASP: Instance-Specific Language Models for Combinatorial Construction via Online Imitation Learning

arXiv:2607. 28135v1 Announce Type: new Abstract: Machine learning for combinatorial optimization typically relies on neural constructors trained via reinforcement learning on large offline datasets for a fixed problem class-incurring high pretraining costs and generalizing poorly outside the training distribution.

By Mohand Mezmaz, Gr\'egoire Danoy

llms reinforcement-learning benchmarks

arXiv Machine Learning

Jul 31

GVR-Coder: A Visual-Feedback Framework for Structured SVG Generation in Complex Document and Meeting Scenarios

arXiv:2607. 28073v1 Announce Type: new Abstract: In demanding professional environments and meeting review scenarios, lengthy text often imposes a high cognitive load.

By Yiming Xu, Jihua Kang, Chunsai Du, Qifan Zhang, Wangqiu Zhou, Yiting Wu, Tianqi Li, Qi Song

agents reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 31

Contrastive Reinforced Policy Optimization via Privileged Self-Distillation

arXiv:2607. 28026v1 Announce Type: new Abstract: Recent advances in post-training Large Language Models (LLMs) increasingly rely on Reinforcement Learning with Verifiable Rewards (RLVR) or On-Policy Self-Distillation (OPSD).

By Xingjian Wu, Junlin Liu, Xingchen Liu, Xuhang Zhu, Jianing Wang, Linsen Guo, Xiaoyu Li, Xuezhi Cao, Xunliang Cai

llms agents reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Jul 31

TAPO: Transition-Aware Policy Optimization for LLM Agents

arXiv:2607. 27973v1 Announce Type: new Abstract: Recently, Reinforcement Learning (RL) has emerged as a crucial paradigm for the post-training of Large Language Model (LLM) agents.

By Cong Li, Peixi Peng, Yisen Zhao, Xinyu Hu, Shudong Liu, Zhan Su, Zhuojian Li

llms agents reinforcement-learning

arXiv Machine Learning

Jul 31

Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning

arXiv:2607. 27968v1 Announce Type: new Abstract: Machine unlearning seeks to selectively remove specific knowledge from trained language models without full retraining, a growing necessity under privacy regulations such as GDPR and the EU AI Act.

By Efstratios Zaradoukas, Davide Gabrielli, Bardh Prenkaj, Gjergji Kasneci

llms reinforcement-learning efficiency benchmarks safety

arXiv Machine Learning

Jul 31

Exact Action Values Are Not Enough: Rollout-Verified Reinforcement Fine-Tuning of a Reasoning Model for Multi-Zone VAV Control

arXiv:2607. 27914v1 Announce Type: new Abstract: Multi-zone variable-air-volume control must balance thermal comfort, indoor air quality, and electricity use across several continuous actuators.

By Takumi Shioda, Kohei Terashima, Tatsuo Nagai

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 31

Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold

arXiv:2607. 27770v1 Announce Type: new Abstract: A single reinforcement-learning run can produce a strong reasoner yet an incomplete teacher: it often amplifies only a subset of the valid solution modes.

By Songshuo Lu, Zhi Chen, Yaohua Tang

reinforcement-learning efficiency

arXiv Machine Learning

Jul 31

Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning

arXiv:2607. 27610v1 Announce Type: new Abstract: Reinforcement learning (RL) finetuning significantly enhances the reasoning capabilities of large language models (LLMs), yet its effectiveness critically depends on selecting prompts of appropriate difficulty for the current policy.

By Haodong Zhu, Yangyang Ren, Yanjing Li, Sheng Xu, Haiguang Liu, Linlin Yang, Baochang Zhang

llms reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jul 31

Real-Time Hard Peak Age-of-Information Safety with No-Regret Learning

arXiv:2607. 27626v1 Announce Type: new Abstract: Safety-critical IoT systems such as industrial closed-loop control, V2X coordination, and remote teleoperation require every sensor's peak Age of Information (peak AoI, also abbreviated PAoI) to stay below a hard per-slot deadline, not merely an average bound.

By Wentao Zhang, Wentao Mo

reinforcement-learning safety

arXiv Machine Learning

Jul 31

Policy Gradient Steering: Interventions from Behavioral Objectives

arXiv:2607. 27574v1 Announce Type: new Abstract: Activation steering has emerged in large language models as a lightweight alternative for dynamically changing a model's behavior at inference time.

By Yoann Poupart, Aur\'elie Beynier, Nicolas Maudet

llms reinforcement-learning

arXiv Machine Learning

Jul 31

SDO: Structure-Aware Data Organization for Efficient LLM Post-Training

arXiv:2607. 27273v1 Announce Type: new Abstract: Post-training of large language models is expensive, and existing efficiency improvements mainly focus on selecting informative samples or designing training schedules.

By Jinliang Gao, Ning Yang, Hai Wang, Baili Xiao, Pin Lyu

llms rag reinforcement-learning fine-tuning

arXiv Machine Learning

Jul 31

PlatformBid: An Auto-Bidding Benchmark from a Unified Advertising Platform's Perspective

arXiv:2607. 27265v1 Announce Type: new Abstract: Real-time bidding is central to computational advertising, comprising three elements: Supply Side Platform (SSP) selling ad impressions, Demand Side Platform (DSP) bidding for advertisers, and Ad Exchange conducting auctions between them.

By Shengtian Yang, Yewen Li, Peng Jiang, Zhiyi Lyu, Bo An, Peng Jiang, Qingpeng Cai, Lei Feng

reinforcement-learning benchmarks

arXiv AI

Jul 31

Property-driven Causal Abstractions for Markov Decision Processes

arXiv:2607. 26787v1 Announce Type: new Abstract: Markov Decision Processes (MDPs) are widely used as decision-making models, commonly specified over factored state spaces through state variables and their valuations.

By Jule Schmidt, Maximilian Weininger, Clemens Dubslaff, David Parker, Nils Jansen

reinforcement-learning benchmarks

arXiv AI

Jul 31

MICA: Multi-granularity Intertemporal Credit Assignment for Long-Horizon Emotional Support Dialogue

arXiv:2603. 06194v3 Announce Type: replace-cross Abstract: Reinforcement learning (RL) for large language models (LLMs) has shown strong performance in single-turn tasks, but extending it to multi-turn interaction remains challenging due to sparse rewards and poor per-turn credit assignment.

By Naifan Zhang, Ruihan Sun, Jinwei Su, Hengjie Yang, Zhengyuan Pan, Zhaohan Chen, Xiaofan Zhang

llms reinforcement-learning

arXiv AI

Jul 31

ATOD: Annealed Turn-Aware On-Policy Distillation for Multi-Turn Agentic Tasks

arXiv:2606. 27814v4 Announce Type: replace Abstract: Training small language-model agents for long-horizon interactive tasks requires both fast imitation and reward-driven improvement.

By Qitai Tan, Zefang Zong, Mo Li, Yipeng Shi, Yang Li, Peng Chen

agents reinforcement-learning efficiency

arXiv AI

Jul 31

TANDEM: Temporal-Aware Neural Detection for Multimodal Hate Speech

arXiv:2601. 11178v3 Announce Type: replace Abstract: Social media platforms are increasingly dominated by long-form multimodal content, where harmful narratives are constructed through a complex interplay of audio, visual, and textual cues.

By Girish A. Koushik, Helen Treharne, Diptesh Kanojia

llms reinforcement-learning multimodal benchmarks safety

arXiv Machine Learning

Jul 31

REFINE-DP: Diffusion Policy Fine-tuning for Humanoid Loco-manipulation via Reinforcement Learning

arXiv:2603. 13707v3 Announce Type: replace-cross Abstract: Humanoid loco-manipulation requires coordinated task-space motion planning with stable loco-manipulation command tracking under complex robot-environment dynamics and long-horizon tasks.

By Zhaoyuan Gu, Yipu Chen, Zimeng Chai, Alfred Cueva, Thong Nguyen, Yifan Wu, Huishu Xue, Minji Kim, Isaac Legene, Fukang Liu, KyoungMok Kim, Ayan Barula, Yongxin Chen, Ye Zhao

diffusion reinforcement-learning robotics fine-tuning

arXiv AI

Jul 31

CaM-Wolf: Causal-Aware Multimodal Agents for Social Deduction Games

arXiv:2607. 26393v1 Announce Type: new Abstract: Social deduction games (SDGs) such as Werewolf have become challenging testbeds for AI agents.

By Zheng Zhang, Nanjie Yao, Jiarui He, Deheng Ye, Peilin Zhao, Hao Wang

llms agents reinforcement-learning multimodal

arXiv Machine Learning

Jul 31

Class-Aware Reinforcement Learning for Counterfactual Explanation Generation

arXiv:2607. 27905v1 Announce Type: new Abstract: Counterfactual explanations (CFEs) enhance the interpretability of black-box models by generating alternative instances with adjusted feature values that achieve a contrastive outcome.

By Muhammad Adil Saleem, Syed Ali Raza, Mary-Anne Williams

reinforcement-learning efficiency safety