arXiv Machine Learning

ACT-JEPA: Novel Joint-Embedding Predictive Architecture for Efficient Policy Representation Learning

arXiv:2501. 14622v5 Announce Type: replace Abstract: Learning efficient representations for decision-making policies is a challenge in imitation learning (IL).

arXiv AI
6d ago

Towards VLA-Dreamer: Refining VLA Behavior Using World Models

The paper proposes a new architecture for Vision‑Language‑Action (VLA) models that improves sample efficiency by training a predictive world model on the vision encoder’s embedding space. It argues that these embeddings are action‑relevant and can be used to predict future states, addressing the lack of an explicit world model in current VLAs. The trained model can also support short‑term planning by sampling actions that lead to desired goal images.

By Parsa Mastouri Kashani, Jan-Gerrit Habekost, Stefan Wermter
arXiv AI
Jul 1

Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding

arXiv:2606. 31232v1 Announce Type: new Abstract: Learning visual world models for planning requires compact latent dynamics that remain sensitive to actions, yet reconstruction-free joint-embedding objectives can collapse to action-insensitive representations.

By Zhenghao Zhang, Yuanxiang Wang, Zhenyu Guan, Yujia Yang, Bingkang Shi, Tianyu Zong, Hongzhu Yi, Guoqing Chao, Xingchen Chen, Tiankun Yang, Chenxi Bao, Tao Yu, Jingjing Zhou, Jungang Xu
arXiv AI
Aug 28

Predicting Consequences and Reinforcing Navigation Policies with Latent World Models

The paper introduces a Latent World Model (LWM) for robot navigation that predicts action‑conditioned latent feature compatibility instead of reconstructing future observations. By exploiting the correlation between spatial proximity and latent feature similarity, the model evaluates action consequences directly in latent space and supports counterfactual training using sampled action sequences. The learned world model can supervise policy learning from unlabeled video and further improve policies via reinforcement learning entirely within the model, eliminating the need for action annotations and additional environment interaction.

By Zengmao Wang, Wei Gao, Shuhan Shen