The paper introduces Hamiltonian Temporal Contrastive Curiosity (HTC²), a method that combines multi‑horizon temporal representation learning with Hamiltonian phase‑space structure to guide exploration in physical control environments. HTC² uses a horizon‑conditioned contrastive critic to learn relationships between future state pairs within an episode, and the resulting representation distance serves as an intrinsic reward. The reward is further modulated by bounded multi‑step Hamiltonian variation, and the framework is integrated with Soft Actor‑Critic and Proximal Policy Optimization, evaluated on six DeepMind Control Suite tasks against several baseline curiosity methods.
Exploration in physical control environments requires agents to identify actions whose consequences reveal previously unexplored aspects of the underlying dynamics. However, many intrinsic motivation...
arXiv:2609.38383v1 Announce Type: cross Abstract: Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without po...
arXiv:2606. 19476v1 Announce Type: cross Abstract: Effective machine learning depends not only on how we model data, but also on what data we choose to collect.
arXiv:2609.05650v1 Announce Type: new Abstract: We propose a reinforcement learning framework in which exploration is driven by intrinsic curiosity, designed for scenarios where environments are non-...
arXiv:2604. 18701v3 Announce Type: replace-cross Abstract: Local prediction-error-based curiosity rewards focus on the current transition without considering the world model's cumulative prediction error across all visited transitions.
arXiv:2605. 00412v3 Announce Type: replace Abstract: World models have recently re-emerged as a central paradigm for embodied intelligence, robotics, autonomous driving, and model-based reinforcement learning.
The paper presents a deep active inference framework for real‑world robotic navigation that combines a diffusion policy with a multiple‑timescale recurrent state‑space model. The diffusion policy generates diverse candidate actions, while the state‑space model predicts long‑horizon outcomes, allowing the system to select actions that minimize expected free energy. Experiments show higher success rates and fewer collisions, especially in exploration‑heavy scenarios, demonstrating the effectiveness of this unified exploration and goal‑directed approach.
arXiv:2607. 07763v1 Announce Type: new Abstract: World models are typically trained to predict discrete-time physical dynamics with a fixed step size baked into the model weights, preventing prediction at variable temporal resolutions.
The paper introduces the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning into distinct latent spaces and dynamics models. A new learning method, Long-Horizon Representation Learning with Weighted Rollout (LoRe), supervises predictions at both levels using exponential horizon weights. Experiments on five goal-conditioned visual control tasks show that Dual-WM improves success rates over strong baselines, especially at longer horizons.
HaM-World introduces a structured world model that combines history-conditioned selective memory with a Soft‑Hamiltonian latent dynamics prior. The model decomposes the latent state into a canonical (q,p) subspace governed by an energy‑derived Hamiltonian vector field and a context subspace c capturing non‑conservative factors, while Mamba selective state‑space memory conditions the transition used for prediction, reward, value estimation, and planning. Across six DeepMind Control Suite tasks, HaM-World achieves top rankings on four tasks, improves average AUC, reduces imagined‑rollout error by 45% on short‑to‑medium horizons, and outperforms baselines under 12 out‑of‑distribution perturbations.
Effective machine learning depends not only on how we model data, but also on what data we choose to collect. While large sequence models have revolutionized data modeling, the problem of automated data selection, or "intrinsic curiosity", remains a significant challenge.
arXiv:2606. 03017v1 Announce Type: cross Abstract: Reward transfer in Inverse Reinforcement Learning (IRL) is unreliable when policies must generalize to unseen combinations of environment dynamics and task goals.