Reinforcement learning

Policy optimisation, reward modelling and RLHF — how models are trained by feedback rather than by labels.

7,178 stories · RSS feed

arXiv AI

Jul 2

Personalization as Inverse Planning: Learning Latent Design Intents for Agentic Slide Generation via Structural Denoising

arXiv:2607. 00407v1 Announce Type: new Abstract: Slide design requires personalizing both deck themes and page layouts.

By Tianci Liu, Zihan Dong, Linjun Zhang, Haoyu Wang, jing Gao, Emre Kiciman, Ranveer Chandra, Wei-Ting Chen

agents reinforcement-learning

arXiv AI

Jul 2

Staleness-Learning Rate Scaling Laws for Asynchronous RLHF

arXiv:2607. 01083v1 Announce Type: cross Abstract: High-throughput RLHF systems often decouple rollout generation from policy optimization, leading to the use of stale rollouts during learner updates.

By Jingwei Song, Haofeng Xu, Jie Xiao, Chengke Bao, Jingwei Shi, Pengbin Feng, Weixun Wang, Yuhang Han, Chuan Wu, Linfeng Zhang, Bill Shi

reinforcement-learning efficiency safety

arXiv AI

Jul 2

Diffusion-GR2: Diffusion Generative Reasoning Re-ranker

arXiv:2607. 01170v1 Announce Type: cross Abstract: Generative reasoning re-rankers achieve strong recommendation accuracy by emitting a chain-of-thought before re-ordering a candidate list, but they are slow at inference: an autoregressive (AR) decoder spends one sequential forward pass per reasoning token, and the reasoning trace far exceeds the ranking it produces.

By Zhuoxuan Zhang (Yang), Kangqi Ni (Yang), Yuhang Chen (Yang), Mingfu Liang (Yang), Xiaohan Wei (Yang), Yunchen Pu (Yang), Fei Tian (Yang), Chonglin Sun (Yang), Frank Shyu (Yang), Adam (Yang), Song, Sandeep Pandey, Luke Simon, Tianlong Chen, Xi Liu

llms diffusion reinforcement-learning fine-tuning efficiency

arXiv Machine Learning

Jul 2

The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology

arXiv:2607. 01033v1 Announce Type: new Abstract: Model organisms (MOs) - language models trained to exhibit undesired or unnatural behaviours - are frequently used as testbeds for evaluating white-box interpretability techniques.

By Andrzej Szablewski, Gabriel Konar-Steenberg, Raffaello Fornasiere, Nikita Menon, Stefan Heimersheim

llms reinforcement-learning fine-tuning benchmarks safety

arXiv AI

Jul 2

Reasoning Up the Instruction Ladder for Controllable Language Models

arXiv:2511. 04694v5 Announce Type: replace-cross Abstract: As large language model (LLM) based systems take on high-stakes roles in real-world decision-making, they must reconcile competing instructions from multiple sources within a single prompt context.

By Zishuo Zheng, Vidhisha Balachandran, Chan Young Park, Faeze Brahman, Sachin Kumar

llms reinforcement-learning benchmarks safety

arXiv AI

Jul 2

Graph-Native Reinforcement Learning Enables Traceable Scientific Hypothesis Generation through Conceptual Recombination

arXiv:2607. 00924v1 Announce Type: new Abstract: Accelerating materials discovery requires AI systems that can generate scientifically valid hypotheses through multi-step, domain-grounded reasoning.

By Subhadeep Pal, Shashwat Sourav, Tirthankar Ghosal, Markus J. Buehler

llms rag reinforcement-learning fine-tuning safety

arXiv Machine Learning

Jul 2

From Pixels to Temporal Correlations: Learning Informative Representations for Reinforcement Learning Pre-training

arXiv:2607. 00811v1 Announce Type: new Abstract: Unsupervised pre-training on large-scale datasets has demonstrated significant potential for improving the sample efficiency and performance of Reinforcement Learning (RL).

By Jinwen Wang, Youfang Lin, Xiaobo Hu, Siyu Yang, Sheng Han, Shuo Wang, Kai Lv

reinforcement-learning

arXiv AI

Jul 2

Heuresis: Search Strategies for Autonomous AI Research Agents Across Quality, Diversity and Novelty

arXiv:2606. 25198v2 Announce Type: replace Abstract: Autonomous AI Research promises to accelerate the scientific progress of machine learning.

By Antonis Antoniades, Deepak Nathani, Ritam Saha, Alfonso Amayuelas, Ivan Bercovich, Zhaotian Weng, Vignesh Baskaran, Kunal Bhatia, William Yang Wang

llms agents reinforcement-learning

arXiv AI

Jul 2

Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration

arXiv:2607. 00033v1 Announce Type: cross Abstract: Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging.

By Xinghao Zhu, Zixi Liu, Shalin Jain, Chenran Li, Milad Noori, Huihua Zhao, John Welsh, Michael Andres Lin, Wei Liu, Tingwu Wang, Xingye Da, Zhengyi Luo, Vishal Kulkarni, Naema Bhatti, Yuke Zhu, Linxi Fan, Bowen Wen, Danfei Xu, Soha Pouya, Yan Chang

reinforcement-learning robotics benchmarks

arXiv AI

Jul 2

A Contextual-Bandit Oversight Game with Two-Sided Informational Asymmetry

arXiv:2607. 00155v1 Announce Type: new Abstract: We study runtime human oversight of an AI agent when private information runs in both directions: the human privately knows her reward function, while the AI privately knows the quality of the action it proposes.

By Yunjin Tong

agents reinforcement-learning robotics

arXiv AI

Jul 2

Language-Critique Imitation Learning from Suboptimal Demonstrations

arXiv:2607. 01225v1 Announce Type: cross Abstract: Prior work on imitation learning from suboptimal demonstrations typically relies on compressed supervision signals such as confidence estimates, discriminator scores, or importance weights.

By Chih-Han Yang, Dai-Jie Wu, Yun-Ping Huang, Ping-Chun Hsieh, Kenneth Marino, Shao-Hua Sun

diffusion reinforcement-learning robotics

arXiv AI

Jul 2

PAPA: Online Personalized Active Preference Alignment

arXiv:2607. 00486v1 Announce Type: cross Abstract: Diffusion models are highly effective at modeling complex data distributions, including images and text.

By Anindya Sarkar, Nasik Muhammad Nafi, Isaac Lyngaas, Muralikrishnan Gopalakrishnan Meena, Yevgeniy Vorobeychik

diffusion reinforcement-learning fine-tuning benchmarks safety

arXiv Machine Learning

Jul 2

Semi-Bandit Learning for Monotone Stochastic Optimization

arXiv:2312. 15427v3 Announce Type: replace Abstract: Stochastic optimization is a widely used approach for optimization under uncertainty, where uncertain input parameters are modeled by random variables.

By Arpit Agarwal, Rohan Ghuge, Viswanath Nagarajan, Zhengjia Zhuo

reinforcement-learning

Berkeley AI Research

Jul 1

2026 BAIR Graduate Showcase

Congratulations to the Berkeley Artificial Intelligence Research (BAIR) Lab class of 2026! This year, BAIR celebrates another remarkable group of Ph.

llms rag agents diffusion computer-vision nlp reinforcement-learning robotics multimodal benchmarks safety

arXiv Machine Learning

Jul 1

Predictable GRPO: A Closed-Form Model of Training Dynamics

arXiv:2606. 30789v1 Announce Type: new Abstract: Group Relative Policy Optimization (GRPO) has become a standard tool for improving the reasoning ability of large language models, yet its training dynamics are still described empirically: reward trajectories are fit with low-parameter functional forms whose constants carry no mechanistic meaning, and hyperparameter choices remain a matter of trial and error.

By Rajat Ghosh, Datta Nimmaturi, Aryan Singhal, Vaishnavi Bhargava, Henry Wong, Johnu George, Debojyoti Dutta

llms reinforcement-learning benchmarks

arXiv AI

Jul 1

InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training

arXiv:2601. 04126v3 Announce Type: replace-cross Abstract: GUI agents that interact with graphical interfaces on behalf of users represent a promising direction for practical AI assistants.

By Ziyun Zhang, Zezhou Wang, Xiaoyi Zhang, Zongyu Guo, Jiahao Li, Bin Li, Yan Lu

llms agents reinforcement-learning

arXiv AI

Jul 1

Lyapunov-Certified Direct Switching Theory for Q-Learning

arXiv:2604. 19569v4 Announce Type: replace-cross Abstract: Q-learning is a fundamental algorithmic primitive in reinforcement learning.

By Donghwan Lee

reinforcement-learning

arXiv AI

Jul 1

HyPOLE: Hyperproperty-Guided Multi-Agent Reinforcement Learning under Partial Observation

arXiv:2606. 30966v1 Announce Type: new Abstract: Formal specification is a powerful tool to guide the learning process and provides significant advantages over reward shaping: (1) mathematical rigor; (2) expressiveness to specify objectives and constraints, and (3) the ability to define tactics to achieve objectives.

By Arshia Rafieioskouei, Tzu-Han Hsu, Matthew Lucas, Borzoo Bonakdarpour

agents reinforcement-learning benchmarks

arXiv AI

Jul 1

Freeform Preference Learning for Robotic Manipulation

arXiv:2606. 32027v1 Announce Type: cross Abstract: Reward design remains a central bottleneck for autonomous robot policy improvement, especially in long-horizon manipulation tasks where sparse success labels provide too little signal and binary preferences collapse many competing notions of quality into one ambiguous signal.

By Marcel Torne, Anubha Mahajan, Abhijnya Bhat, Chelsea Finn

agents computer-vision reinforcement-learning robotics safety

arXiv AI

Jul 1

Disentangling Reasoning Logic to Resolve Explicit Knowledge Conflicts

arXiv:2508. 01273v3 Announce Type: replace Abstract: Explicit knowledge conflicts, occurring when retrieved contexts contain contradictory information, pose a fundamental challenge for Large Language Models (LLMs) as they integrate increasingly diverse data sources.

By Xianda Zheng, Zijian Huang, Meng-Fen Chiang, Jiamou Liu, Yuan Fang, Michael Witbrock, Kaiqi Zhao

llms reinforcement-learning

Get the daily briefing