arXiv Machine Learning October 3, 2026 By Scott W. Viteri (Stanford University), Laura Gomezjurado Gonzalez (Stanford University), Clark Barrett (Stanford University)

When Do Intrinsic Rewards Lead to Exploration?

Read the original on arXiv Machine Learning →

The paper investigates when intrinsic rewards effectively drive exploration in reinforcement learning. It introduces a formal criterion that evaluates policies based on the counterfactual information they acquire, comparing how well their histories can replace experience from alternative policies. Using a simple environment, the authors show that common intrinsic reward objectives—count-based, prediction-error, empowerment, and information-gain—can lead to Pareto-suboptimal exploration under this criterion, and they propose conditions and a new objective that better align with optimal exploration.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 2

Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying

arXiv:2606. 00151v1 Announce Type: cross Abstract: In reinforcement learning (RL), agents benefit from exploration only because they repeatedly encounter similar states: trying different actions can improve performance or reduce uncertainty; without such retries, a greedy policy is optimal.

By Soichiro Nishimori, Paavo Parmas, Sotetsu Koyamada, Tadashi Kozuno, Toshinori Kitamura, Shin Ishii, Yutaka Matsuo

agents reinforcement-learning benchmarks

arXiv AI

Sep 10

Efficient Exploration Is Enough

arXiv:2609.07575v1 Announce Type: cross Abstract: This work introduces an alternative view of efficient exploration and studies its theoretical and empirical implications in the absence of extrinsic...

By Mikel Malag\'on, Jon Vadillo, Josu Ceberio, Michael Bowling, Jose A. Lozano

agents

arXiv AI

Aug 3

Explore Beyond the Boundary Using Entropic Information

arXiv:2607. 29419v1 Announce Type: cross Abstract: In reinforcement learning, exploration with sparse and delayed rewards presents a significant challenge due to the limited feedback available for guiding the learning process.

By Bumgeun Park, Donghwan Lee

agents reinforcement-learning

arXiv AI

Jun 19

Can In-Context Learning Support Intrinsic Curiosity?

arXiv:2606. 19476v1 Announce Type: cross Abstract: Effective machine learning depends not only on how we model data, but also on what data we choose to collect.

By Eric Elmoznino, Sangnie Bhardwaj, Johannes von Oswald, Rajai Nasser, Blaise Ag\"uera y Arcas, Jo\~ao Sacramento, Rif A. Saurous, Guillaume Lajoie

llms agents reinforcement-learning safety

arXiv AI

Jun 18

UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning

arXiv:2606. 19328v1 Announce Type: cross Abstract: Preference-based RL provides an approach to learning reward models from pairwise comparisons of behaviors, bypassing the need for explicit reward design.

By Mohamed Nabail, Leo Cheng, Jingmin Wang, Nicholas Rhinehart

reinforcement-learning benchmarks

arXiv AI

Jul 10

Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning

arXiv:2607. 08647v1 Announce Type: cross Abstract: As autonomous agents are increasingly deployed across diverse operational contexts, aligning their behavior with human intent demands reward functions that remain robust to such changes rather than overfitting to any single environment.

By Ali Larian, Qian Lin, Chang Zong Wu, Daniel S. Brown

agents reinforcement-learning multimodal

One email a morning, machine-written

Related stories

Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying

Efficient Exploration Is Enough

Explore Beyond the Boundary Using Entropic Information

Can In-Context Learning Support Intrinsic Curiosity?

UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning

Multi-Modal, Multi-Environment Machine Teaching for Robust Reward Learning