OpenAI Blog
Apr 5, 2018
Retro Contest
We’re launching a transfer learning contest that measures a reinforcement learning algorithm’s ability to generalize from previous experience.
Quantifying generalization in reinforcement learning
We’re releasing CoinRun, a training environment which provides a metric for an agent’s ability to transfer its experience to novel situations and has already helped clarify a longstanding puzzle in reinforcement learning. CoinRun strikes a desirable balance in complexity: the environment is simpler than traditional platformer games like Sonic the Hedgehog but still poses a worthy generalization challenge for state of the art algorithms.
Related stories
arXiv AI
Sep 10
Efficient Exploration Is Enough
arXiv:2609.07575v1 Announce Type: cross Abstract: This work introduces an alternative view of efficient exploration and studies its theoretical and empirical implications in the absence of extrinsic...
arXiv AI
Jun 2
Certificate-Guided Evaluation of Reinforcement Learning Generalization
arXiv:2606. 00840v1 Announce Type: new Abstract: This work presents a logic-driven framework to evaluate the performance of reinforcement learning (RL) algorithms in their ability to generalize to unseen tasks.
arXiv AI
Jul 14
Training on Irrelevant States Implies Data Augmentation: Generalization in Contextual MDPs
arXiv:2410. 03565v4 Announce Type: replace-cross Abstract: In the zero-shot policy transfer (ZSPT) setting for contextual Markov decision processes (CMDP), agents train on a fixed, finite set of contexts and must generalize to new ones.
arXiv Machine Learning
1d ago
When Do Intrinsic Rewards Lead to Exploration?
The paper investigates when intrinsic rewards effectively drive exploration in reinforcement learning. It introduces a formal criterion that evaluates policies based on the counterfactual information they acquire, comparing how well their histories can replace experience from alternative policies. Using a simple environment, the authors show that common intrinsic reward objectives—count-based, prediction-error, empowerment, and information-gain—can lead to Pareto-suboptimal exploration under this criterion, and they propose conditions and a new objective that better align with optimal exploration.
arXiv AI
3d ago
ASH: Agents that Self-Hone in Long-Horizon Worlds
arXiv:2605.14211v4 Announce Type: replace Abstract: Long-horizon visuomotor tasks remain a fundamental challenge in AI, as current methods rely on hand-engineered rewards or action-labeled demonstrat...
OpenAI Blog
Jul 4, 2018
Learning Montezuma’s Revenge from a single demonstration
We’ve trained an agent to achieve a high score of 74,500 on Montezuma’s Revenge from a single human demonstration, better than any previously published result. Our algorithm is simple: the agent plays a sequence of games starting from carefully chosen states from the demonstration, and learns from them by optimizing the game score using PPO, the same reinforcement learning algorithm that underpins OpenAI Five.
arXiv AI
Jun 9
An Information-Theoretic Definition for Open-Ended Learning
arXiv:2606. 08369v1 Announce Type: cross Abstract: A growing body of work points to the great promise of AI systems that can continually expand their capabilities as they operate in an open-ended environment.
arXiv AI
Jun 2
MindGames Arena Generalization Track: In2AI Solution with Delayed Per-Step Reward Attribution
arXiv:2606. 00017v1 Announce Type: new Abstract: Training language model agents for multi-agent strategic interaction presents a core difficulty: the quality of any action may depend on future events that never materialize, on moves that violate game rules, or on decisions made by other players.
OpenAI Blog
Apr 18, 2018
Evolved Policy Gradients
We’re releasing an experimental metalearning approach called Evolved Policy Gradients, a method that evolves the loss function of learning agents, which can enable fast training on novel tasks. Agents trained with EPG can succeed at basic tasks at test time that were outside their training regime, like learning to navigate to an object on a different side of the room from where it was placed during training.
arXiv Machine Learning
Aug 11
Regret, equilibrium, and learning in games: A guided tour
arXiv:2608. 09389v1 Announce Type: cross Abstract: This note aims to serve as an entry point to the literature on learning in games, a topic with significant theoretical appeal and a wide range of applications -- from machine learning and data science to economics and beyond.
arXiv AI
Jun 9
ASH: Agents that Self-Hone via Embodied Learning
arXiv:2605. 14211v3 Announce Type: replace Abstract: Long-horizon embodied tasks remain a fundamental challenge in AI, as current methods rely on hand-engineered rewards or action-labeled demonstrations, neither of which scales.