Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

6,940 stories · RSS feed

arXiv Machine Learning

Aug 21

FAR-DPO: Feasibility-Aware and Robust Direct Preference Optimization for Cyclic Peptide Design

arXiv:2608. 19808v1 Announce Type: new Abstract: Cyclic peptides are emerging as promising molecular scaffolds in drug discovery due to their high binding affinity and structural stability.

By Guofeng Zhang, Rong Han, Xiaoyu Wang, Zhiyun Li, Zongbo Han, Xiaohong Liu, Guangyu Wang

reinforcement-learning benchmarks

arXiv Machine Learning

Aug 21

End-to-end Early Classification of Time Series in Non-Stationary Environments

arXiv:2608. 20044v1 Announce Type: new Abstract: Early Classification of Time Series (ECTS) requires making accurate decisions as early as possible in inherently online and evolving environments.

By Aur\'elien Renault, Alexis Bondu, Antoine Cornu\'ejols, Vincent Lemaire

reinforcement-learning benchmarks

arXiv AI

Aug 21

Truncate Bad, Upweight Good: BoN-Style Distillation via Rank-Based Classification

arXiv:2608. 19748v1 Announce Type: cross Abstract: Inference-time selection methods, such as Best-of-N, improve generation by sampling a pool of candidates and selecting the top-ranked completion according to a reward model.

By Yarin Bar, Yaniv Romano

reinforcement-learning efficiency safety

arXiv AI

Aug 21

EnvHarness: Awakening Static Worlds for Agent Learning

arXiv:2608. 19880v1 Announce Type: new Abstract: LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves.

By Chengsong Huang, Zifeng Wang, Rujun Han, Jun Yan, Yanfei Chen, Zoey CuiZhu, Ke Jiang, Peng Xia, Han Yu, Yufan Zhuang, Yifei Ming, Jiaqi Pan, Bhavana Dalvi Mishra, Jiaxin Huang, Burak Gokturk, Tomas Pfister, Chen-Yu Lee

llms agents reinforcement-learning benchmarks

arXiv Machine Learning

Aug 21

Higher Resolution, Better Generalization: Unlocking Visual Scaling in Deep Reinforcement Learning

arXiv:2605. 10546v2 Announce Type: replace Abstract: Pixel-based deep reinforcement learning agents are typically trained on heavily downsampled visual observations, a convention inherited from early benchmarks rather than grounded in principled design.

By Raphael Trumpp, \"Omer Veysel \c{C}a\u{g}atan, Bar{\i}\c{s} Akg\"un, Marco Caccamo

agents reinforcement-learning benchmarks

arXiv Computation and Language

Aug 21

Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving

arXiv:2608. 20129v1 Announce Type: cross Abstract: Autonomous vehicles require robust perception and decision-making capabilities to operate in diverse and unseen scenarios.

By Mehdi Azarafza, Faezeh Pasandideh, Ali Ehteshami Bejnordi, Stefan Henkler, Achim Rettberg

llms agents reinforcement-learning robotics multimodal safety

arXiv Machine Learning

Aug 21

Asymptotic Theory for IV-Based Reinforcement Learning with Potential Endogeneity

arXiv:2103. 04021v4 Announce Type: replace-cross Abstract: In the standard data analysis framework, data is collected (once and for all), and then data analysis is carried out.

By Jin Li, Ye Luo, Zigan Wang, Xiaowei Zhang

reinforcement-learning safety

arXiv AI

Aug 21

Concentrated Liquidity Provision: a Reinforcement Learning Perspective

arXiv:2608. 19389v1 Announce Type: cross Abstract: Automated market makers (AMMs) are a cornerstone of decentralised finance (DeFi).

By Georgios Chionas, Charalampos Kleitsikas, Stefanos Leonardos, Leandro S\'anchez-Betancourt, Carmine Ventre

agents reinforcement-learning benchmarks

arXiv AI

Aug 21

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation

arXiv:2608. 20256v1 Announce Type: new Abstract: Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones.

By Gijs Kassenaar, Zhao Yang, Vincent Fran\c{c}ois-Lavet

llms reinforcement-learning benchmarks

arXiv Machine Learning

Aug 21

Towards Formalizing Reinforcement Learning Theory: A Robbins-Siegmund Approach

arXiv:2511. 03618v2 Announce Type: replace Abstract: In this paper, we formalize the almost sure convergence of $Q$-learning and linear temporal difference (TD) learning with Markovian samples using the Lean 4 theorem prover based on the Mathlib library.

By Shangtong Zhang

reinforcement-learning

arXiv Computer Vision

Aug 21

Video Evidence to Reasoning Efficient Video Understanding via Explicit Evidence Grounding

arXiv:2601. 07761v2 Announce Type: replace Abstract: Large Vision-Language Models (LVLMs) face a fundamental dilemma in video reasoning: they are caught between the prohibitive computational costs of verbose reasoning and the hallucination risks of efficient, ungrounded approaches.

By Yanxiang Huang, Guohua Gao, Zhaoyang Wei

llms reinforcement-learning multimodal benchmarks safety

arXiv Machine Learning

Aug 21

ReAugment: Model Zoo-Guided RL for Few-Shot Time Series Augmentation and Forecasting

arXiv:2409. 06282v5 Announce Type: replace Abstract: Time series forecasting, particularly in few-shot learning scenarios, is challenging due to the limited availability of high-quality training data.

By Haochen Yuan, Yutong Wang, Yihong Chen, Yunbo Wang, Xiaokang Yang

reinforcement-learning

arXiv AI

Aug 21

DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values

arXiv:2509. 08022v3 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) with diverse human values is essential for safe and effective deployment, yet existing benchmarks often overlook cultural and demographic variation.

By Yao Liang, Dongcheng Zhao, Feifei Zhao, Guobin Shen, Yuwei Wang, Dongqi Liang, Yi Zeng

llms reinforcement-learning fine-tuning benchmarks safety

arXiv Computer Vision

Aug 21

Swift-Image: Exploring the Performance Frontier of Compact Unified Image Generation Models

arXiv:2608. 20334v1 Announce Type: new Abstract: We present Swift-Image, a compact unified model for text-to-image generation, single-image editing, and multi-image editing.

By Taihang Hu, Zhao Wang, Zuan Gao, Tao Liu, Hao Yan, Zhengze Xu, Yuhang Yu, Yongchao Du, Xingjian Wang, Jun Zheng, Qinye Zhou, Zhengrui Chen, Chao Lin, Yefeng Shen, Zhengtao Wu, Ge Wu, Xiaoli Xu, Denghui Yang, Huayu Zhang, Mingzhou Zhang, Mengting Chen

diffusion reinforcement-learning efficiency

arXiv Computer Vision

Aug 21

RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation

arXiv:2608. 20208v1 Announce Type: new Abstract: Offline reinforcement learning improves robotic policies using previously collected data without further environment interaction.

By Shaoxuan Wang, Guangting Zheng, Rui Huang, Zhipeng Tang, Sha Zhang, Jiajun Deng, Yanyong Zhang

diffusion reinforcement-learning robotics efficiency benchmarks

arXiv AI

Aug 21

PEA-DPO: Perception-Enhanced Alignment Direct Preference Optimization for MLLMs Alignment

arXiv:2608. 19598v1 Announce Type: cross Abstract: Direct Preference Optimization (DPO) has emerged as an effective approach for aligning large language models (LLMs) with human preferences.

By Jiawei Feng, Jiancan Wu, Xingyu Zhu, Junkang Wu, Xiang Wang, Xiangnan He

llms reinforcement-learning multimodal benchmarks safety

arXiv AI

Aug 21

Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning

arXiv:2608. 19836v1 Announce Type: cross Abstract: Probabilistic shielding is a technique for safe reinforcement learning (RL).

By Astrid Horn Brorholt (Aalborg University, Aalborg, Denmark), Maris F. L. Galesloot (Radboud University, Nijmegen, Netherlands), Nils Jansen (Radboud University, Nijmegen, Netherlands), Kim Guldstrand Larsen (Aalborg University, Aalborg, Denmark), Christian Schilling (Aalborg University, Aalborg, Denmark)

agents reinforcement-learning safety

arXiv Machine Learning

Aug 21

LODESTAR: Robust Entropy-Based Answer Selection in Retrieval-Augmented Generation for Question Answering -- Directing Frozen-LLM Entropy with a Reinforcement-Learned Prompt Polarizer under Misleading Passages

arXiv:2608. 11922v2 Announce Type: replace-cross Abstract: Predictive-distribution entropy is a strong answer-selection rule in retrieval-augmented generation (RAG) for question answering: across five QA benchmarks, selecting the answer a frozen respondent LLM produces with the lowest answer-token entropy lifts mean $F_1$ from 0.

By Hung-Chun Hsu, Po-Jen Ko, Che-Cheng Wu, Li-Yang Chang, Chuan-Ju Wang

llms rag nlp reinforcement-learning benchmarks

arXiv Machine Learning

Aug 21

The Price of Hidden Curvature: Improved Lower Bounds for Bandit Convex Optimization

arXiv:2607. 18652v3 Announce Type: replace-cross Abstract: We establish improved lower bounds on the minimax expected regret of stochastic bandit convex optimization for $1$-Lipschitz functions on the $d$-dimensional Euclidean ball.

By Nived Rajaraman, Yanjun Han

llms reinforcement-learning

arXiv Machine Learning

Aug 21

RIPE++: Reinforced Keypoint Learning from Positive Pairs Only

arXiv:2608. 19693v1 Announce Type: cross Abstract: Sparse keypoint extraction and matching underpin core tasks in geometric computer vision, including structure-from-motion, visual SLAM, augmented reality, and medical image registration.

By Johannes K\"unzel, Peter Eisert, Anna Hilsmann

computer-vision reinforcement-learning benchmarks

Get the daily briefing