arXiv Machine Learning
Oct 28

Multi-Horizon Future Discovery for Physics-Informed Exploration

The paper introduces Hamiltonian Temporal Contrastive Curiosity (HTC²), a method that combines multi‑horizon temporal representation learning with Hamiltonian phase‑space structure to guide exploration in physical control environments. HTC² uses a horizon‑conditioned contrastive critic to learn relationships between future state pairs within an episode, and the resulting representation distance serves as an intrinsic reward. The reward is further modulated by bounded multi‑step Hamiltonian variation, and the framework is integrated with Soft Actor‑Critic and Proximal Policy Optimization, evaluated on six DeepMind Control Suite tasks against several baseline curiosity methods.

By Shuwei Wang, Qiang Guan, Brian Qiu, Haoxiang Huang, Haoxi Wang, Chenyu Li, Bo Xu, Sen Cui