Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,242 stories · RSS feed

arXiv AI

Jun 18

Learning from Own Solutions: Self-Conditioned Credit Assignment for Reinforcement Learning with Verifiable Rewards

arXiv:2606. 18810v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in training LLMs for reasoning tasks, but representative methods such as GRPO assign uniform credit across all tokens, wasting gradient on routine tokens while under-crediting pivotal reasoning steps.

By Yingyu Shan, Yuhang Guo, Zihao Cheng, Zeming Liu, Xiangrong Zhu, Xinyi Wang, Jiashu Yao, Wei Lin, Hongru Wang, Heyan Huang

llms agents reinforcement-learning efficiency benchmarks

arXiv AI

Jun 18

SFT Overtraining Predicts Rank Inversion via Entropy Collapse Under RLVR

arXiv:2606. 18487v1 Announce Type: cross Abstract: The standard heuristic of selecting the SFT checkpoint with the highest pass@1 for GRPO can fail when SFT compresses the rollout distribution.

By Siddharth Aphale, Kelly Liu

reinforcement-learning fine-tuning

arXiv AI

Jun 18

Generating Natural and Expressive Robot Gestures through Iterative Reinforcement Learning with Human Feedback using LLMs

arXiv:2606. 18747v1 Announce Type: cross Abstract: Expressive gestures are essential for natural and effective communication, complementing speech when verbal cues alone are insufficient (e.

By Chris Lee, Flora Salim, Benjamin Tag, Francisco Cruz

llms reinforcement-learning robotics

arXiv AI

Jun 18

Bayesian Anytime Pareto Set Identification for Multi-Objective Multi-Armed Bandits

arXiv:2606. 18785v1 Announce Type: cross Abstract: Identifying Pareto optimal solutions is critical to support multi-objective decision-making.

By Lennert Saerens, Bram Silue, Eleni Litsa, Peter Vrancx, Pieter Libin

reinforcement-learning benchmarks

arXiv AI

Jun 18

MemRerank: Preference Memory for Personalized Product Reranking

arXiv:2603. 29247v3 Announce Type: replace-cross Abstract: LLM-based shopping agents increasingly rely on long purchase histories and multi-turn interactions for personalization, yet naively appending raw history to prompts is often ineffective due to noise, length, and relevance mismatch.

By Zhiyuan Peng, Xuyang Wu, Huaixiao Tou, Yi Fang, Yu Gong

llms rag agents reinforcement-learning benchmarks

arXiv Machine Learning

Jun 18

The Reward Was in Your Data All Along: Correcting Flow Matching with Discriminator-Guided RL

arXiv:2606. 19162v1 Announce Type: new Abstract: Score- and flow-matching models often rely on preference-based reinforcement learning for two purposes: aligning with subjective preferences and, surprisingly, recovering properties such as visual realism and coherent object structure that matching-based training is intended to learn from the data itself.

By Nicolas Beltran-Velez, Felix Friedrich, Zhang Xiaofeng, Reyhane Askari-Hemmat, Xiaochuang Han, Adriana Romero-Soriano, Michal Drozdzal

diffusion reinforcement-learning safety

arXiv Machine Learning

Jun 18

Hierarchical Planning with Latent World Models

arXiv:2604. 03208v2 Announce Type: replace Abstract: World models are a promising path to zero-shot embodied control through planning.

By Wancong Zhang, Basile Terver, Artem Zholus, Soham Chitnis, Harsh Sutaria, Mido Assran, Randall Balestriero, Amir Bar, Adrien Bardes, Yann LeCun, Nicolas Ballas

diffusion reinforcement-learning robotics

arXiv AI

Jun 18

Structured Representation Learning with Locally Linear Embeddings and Adaptive Feature Fusion

arXiv:2606. 18469v1 Announce Type: cross Abstract: Neuroscientific research has revealed that the brain encodes complex behaviors by leveraging structured, low-dimensional manifolds and dynamically fusing multiple sources of information through adaptive gating mechanisms.

By Somjit Nath, Jackson J Cone, Derek Nowrouzezahrai, Samira Ebrahimi Kahou

rag reinforcement-learning benchmarks

arXiv AI

Jun 18

Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation

arXiv:2606. 18594v1 Announce Type: cross Abstract: In real-world reinforcement learning (RL), the choice of action space can play a key role in shaping motion smoothness, safety, and overall task performance.

By Seyed Alireza Azimi, Homayoon Farrahi, Abhishek Naik, Colin Bellinger, A. Rupam Mahmood

reinforcement-learning robotics benchmarks safety

arXiv AI

Jun 18

Robust Regularized Policy Iteration under Transition Uncertainty

arXiv:2603. 09344v3 Announce Type: replace Abstract: Offline reinforcement learning (RL) enables data-efficient and safe policy learning without online exploration, but its performance often degrades under distribution shift.

By Hongqiang Lin, Zhenghui Fu, Weihao Tang, Pengfei Wang, Yiding Sun, Qixian Huang, Dongxu Zhang

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 18

Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization

arXiv:2606. 18961v1 Announce Type: new Abstract: Protein language models (PLMs) have emerged as powerful tools for controllable biomolecular design, yet their post-training adaptation typically relies on costly wet-lab validation or curated preference datasets.

By Lanqing Li, Shentong Mo, Yang Yu, Pheng-Ann Heng

llms reinforcement-learning fine-tuning

arXiv Machine Learning

Jun 18

Reinforcement Learning for Accelerated Aerodynamic Shape Optimisation

arXiv:2507. 17786v2 Announce Type: replace Abstract: We introduce a reinforcement learning (RL) based adaptive optimization algorithm for aerodynamic shape optimization focused on dimensionality reduction.

By Florian Sobieczky, Alfredo Lopez, Erika Dudkin, Christopher Lackner, Matthias Hochsteger, Bernhard Scheichl, Helmut Sobieczky

reinforcement-learning

arXiv AI

Jun 18

Reinforcement Learning Foundation Models Should Already Be A Thing

arXiv:2606. 18812v1 Announce Type: cross Abstract: Foundation models for language and vision are powered by internet-scale data, while structured domains (tabular prediction, time-series forecasting, graph learning, reinforcement learning) are not.

By Abdelrahman Zighem, Jill-J\^enn Vie

llms reinforcement-learning benchmarks

arXiv AI

Jun 18

ThinkDeception: A Progressive Reinforcement Learning Framework for Interpretable Multimodal Deception Detection

arXiv:2606. 18988v1 Announce Type: new Abstract: Multimodal deception detection is critical for identifying fraudulent intentions, yet existing approaches predominantly rely on end to end black--box paradigms.

By Jinhao Song, Shan Liang, Yiqun Yue, Zhuhuayang Zhang, Tianqi Gao

llms reinforcement-learning multimodal benchmarks safety

arXiv AI

Jun 18

STARE: Surprisal-Guided Token-Level Advantage Reweighting for Policy Entropy Stability

arXiv:2606. 19236v1 Announce Type: cross Abstract: Reinforcement Learning with Verifiable Rewards algorithms like GRPO have emerged as the dominant post-training paradigm for complex reasoning in LLMs, yet commonly suffer from policy entropy collapse during training.

By Haipeng Luo, Qingfeng Sun, Songli Wu, Can Xu, Wenfeng Deng, Han Hu, Yansong Tang

llms agents reinforcement-learning

arXiv AI

Jun 18

UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning

arXiv:2606. 19328v1 Announce Type: cross Abstract: Preference-based RL provides an approach to learning reward models from pairwise comparisons of behaviors, bypassing the need for explicit reward design.

By Mohamed Nabail, Leo Cheng, Jingmin Wang, Nicholas Rhinehart

reinforcement-learning benchmarks

arXiv Machine Learning

Jun 18

Context-Aware Optimization of Follow-Up Intervals for Type 2 Diabetes Care Using Markov Decision Processes

arXiv:2606. 19092v1 Announce Type: cross Abstract: Chronic disease management relies on regular patient-provider interactions to follow-up on disease progression and control.

By Parisa Lotfibagha, Kristen Miller, William J. Gallagher, Elizabeth B. Selden, Muge Capan

reinforcement-learning benchmarks

arXiv AI

Jun 18

ProductConsistency: Improving Product Identity Preservation in Instruction-Based Image Editing via SFT and RL

arXiv:2606. 19103v1 Announce Type: cross Abstract: Recent advances in instruction-based image editing have enabled models to perform complex visual edits from natural language instructions.

By Mukund Khanna, Raj Singh Yadav, Kunal Singh

computer-vision reinforcement-learning fine-tuning benchmarks

arXiv Machine Learning

Jun 18

REVES: REvision and VErification--Augmented Training for Test-Time Scaling

arXiv:2606. 18910v1 Announce Type: new Abstract: Test-time scaling via sequential revision has emerged as a powerful paradigm for enhancing Large Language Model (LLM) reasoning.

By Yuanxin Liu, Ruida Zhou, Xinyan Zhao, Amr Sharaf, Hongzhou Lin, Arijit Biswas, Mohammad Ghavamzadeh, Zhaoran Wang, Mingyi Hong

llms reinforcement-learning benchmarks

arXiv AI

Jun 18

InfoPO: Information-Driven Policy Optimization for User-Centric Agents

arXiv:2603. 00656v2 Announce Type: replace Abstract: Real-world user requests to LLM agents are often underspecified.

By Fanqi Kong, Jiayi Zhang, Mingyi Deng, Chenglin Wu, Yuyu Luo, Bang Liu

llms agents reinforcement-learning

Get the daily briefing