arXiv Machine Learning By Bonnie Li

Contrastive World Models

Read the original on arXiv Machine Learning →

Contrastive World Models propose a new method for learning latent dynamics without pixel reconstruction. By replacing observation reconstruction with a Deep InfoMax-like objective that maximizes mutual information between state-action sequences and local patch features of future observations, the approach encourages state representations to retain predictive information while ignoring visually irrelevant details. Experiments show that this method matches existing baselines in simple settings and significantly outperforms them when distractors or natural video backgrounds are present, while also training more efficiently by eliminating the pixel decoder.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computer Vision
Sep 28

StarWM: Self-Supervised Trained Attention Routing for Robust World Models

StarWM introduces a self‑supervised attention routing mechanism that selectively applies reconstruction only to dynamically relevant regions of visual input. By combining a cross‑attention module with a dual‑stream decoder and stop‑gradient barriers, it balances faithful environmental dynamics capture with abstraction of irrelevant content. Experiments on DeepMind Control show that StarWM outperforms both reconstruction‑based and reconstruction‑free baselines, especially under distractor conditions, and preserves state attributes over long‑horizon imagination.

By Zeqiang Zhang, Fabian Wurzberger, Maximilian Otte, Daniel Schmid, Sebastian Gottwald, Arne Peter Raulf, Daniel Alexander Braun
arXiv AI
Nov 6

MAC-v2: Long-horizon model-based value expansion from pixels

MAC‑v2 is a new framework for long‑horizon model‑based value expansion from pixel observations in offline reinforcement learning. It trains a reconstruction‑free latent world model that learns representations through prediction and control objectives, avoiding the need to encode control‑irrelevant visual details. By using action chunking, MAC‑v2 reduces recursive model predictions, mitigating error accumulation and achieving strong performance on visual benchmarks such as OGBench, MMBench2, Robomimic, and LIBERO‑10.

By Kwanyoung Park, Sergey Levine
arXiv Computer Vision
Nov 5

Motion-guided Contrastive Regularization For Latent World Models

The paper introduces motion-guided contrastive regularization to improve JEPA-style world models by preventing static collapse, where encoders overemphasize episode-specific static content. By comparing predicted latent transitions with transitions to candidate observations within the same episode and using optical-flow similarities as soft contrastive targets, the method encourages representations that preserve manipulation-relevant state changes. Experiments on OGBench-Cube and LIBERO-Object show significant gains in planning success while maintaining performance on other visual-control benchmarks.

By Yiwen Chen, Qiqi Hou, Jisoo Jeong, Yinhao Zhu, Huaizu Jiang, Amir Said