OpenAI Blog
Apr 5, 2018
Retro Contest
We’re launching a transfer learning contest that measures a reinforcement learning algorithm’s ability to generalize from previous experience.
Quantifying generalization in reinforcement learning
We’re releasing CoinRun, a training environment which provides a metric for an agent’s ability to transfer its experience to novel situations and has already helped clarify a longstanding puzzle in reinforcement learning. CoinRun strikes a desirable balance in complexity: the environment is simpler than traditional platformer games like Sonic the Hedgehog but still poses a worthy generalization challenge for state of the art algorithms.
Related stories
arXiv AI
Jun 2
Certificate-Guided Evaluation of Reinforcement Learning Generalization
arXiv:2606. 00840v1 Announce Type: new Abstract: This work presents a logic-driven framework to evaluate the performance of reinforcement learning (RL) algorithms in their ability to generalize to unseen tasks.
arXiv AI
Jul 14
Training on Irrelevant States Implies Data Augmentation: Generalization in Contextual MDPs
arXiv:2410. 03565v4 Announce Type: replace-cross Abstract: In the zero-shot policy transfer (ZSPT) setting for contextual Markov decision processes (CMDP), agents train on a fixed, finite set of contexts and must generalize to new ones.
OpenAI Blog
Jul 4, 2018
Learning Montezuma’s Revenge from a single demonstration
We’ve trained an agent to achieve a high score of 74,500 on Montezuma’s Revenge from a single human demonstration, better than any previously published result. Our algorithm is simple: the agent plays a sequence of games starting from carefully chosen states from the demonstration, and learns from them by optimizing the game score using PPO, the same reinforcement learning algorithm that underpins OpenAI Five.
arXiv AI
Jun 9
An Information-Theoretic Definition for Open-Ended Learning
arXiv:2606. 08369v1 Announce Type: cross Abstract: A growing body of work points to the great promise of AI systems that can continually expand their capabilities as they operate in an open-ended environment.
arXiv AI
Jun 2
MindGames Arena Generalization Track: In2AI Solution with Delayed Per-Step Reward Attribution
arXiv:2606. 00017v1 Announce Type: new Abstract: Training language model agents for multi-agent strategic interaction presents a core difficulty: the quality of any action may depend on future events that never materialize, on moves that violate game rules, or on decisions made by other players.
OpenAI Blog
Apr 18, 2018
Evolved Policy Gradients
We’re releasing an experimental metalearning approach called Evolved Policy Gradients, a method that evolves the loss function of learning agents, which can enable fast training on novel tasks. Agents trained with EPG can succeed at basic tasks at test time that were outside their training regime, like learning to navigate to an object on a different side of the room from where it was placed during training.
arXiv Machine Learning
Aug 11
Regret, equilibrium, and learning in games: A guided tour
arXiv:2608. 09389v1 Announce Type: cross Abstract: This note aims to serve as an entry point to the literature on learning in games, a topic with significant theoretical appeal and a wide range of applications -- from machine learning and data science to economics and beyond.
arXiv AI
Jun 9
ASH: Agents that Self-Hone via Embodied Learning
arXiv:2605. 14211v3 Announce Type: replace Abstract: Long-horizon embodied tasks remain a fundamental challenge in AI, as current methods rely on hand-engineered rewards or action-labeled demonstrations, neither of which scales.
arXiv Machine Learning
Jun 17
EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning
arXiv:2606. 17680v1 Announce Type: new Abstract: Reinforcement learning (RL) has emerged as a powerful paradigm for training Large Language Models (LLMs) as agents.
arXiv Machine Learning
Jun 18
Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agents
arXiv:2606. 18537v1 Announce Type: new Abstract: Humans often acquire new skills by observing others, since observed behaviors implicitly reveal how to act in an environment.
arXiv Machine Learning
Jun 8
Self-evolving LLM agents with in-distribution Optimization
arXiv:2606. 07367v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently emerged as powerful controllers for interactive agents in complex environments, yet training them to perform reliable long-horizon decision making remains a fundamental challenge.