Latent evolving World Action Model
arXiv:2609.27455v1 Announce Type: new Abstract: World Action Models (WAMs) jointly model action generation and environment dynamics and are mostly built on pretrained Video Diffusion Models (VDMs). I...
arXiv:2609.27455v1 Announce Type: new Abstract: World Action Models (WAMs) jointly model action generation and environment dynamics and are mostly built on pretrained Video Diffusion Models (VDMs). I...
arXiv:2608. 11605v1 Announce Type: new Abstract: World Action Models (WAMs) couple future visual prediction with robot action generation, enabling policies to model how the physical world evolves during interaction.
MT‑WAM enhances the Fast‑WAM framework by adding complementary supervision for future 2‑D point trajectories and visual features while keeping the original training objectives. A lightweight dual‑stream branch and structured attention mask isolate motion‑specific processing, and motion‑stream tokens provide additional dynamics cues to the action expert. During inference, MT‑WAM skips future‑video prediction, using cached video and motion information to achieve higher success rates on LIBERO, LIBERO‑Plus, RoboTwin 2.0 Clean2Rand, and several real‑world tasks.
SaiVLA-JEPA is a family of vision‑language‑action policies that condition on frozen V‑JEPA features and use lightweight auxiliary heads to predict future latent representations during training. Experiments on 10,030 LIBERO‑Plus episodes show that JEPA conditioning improves success rates to about 82.5% versus 71.6% without JEPA, but adding future supervision does not yield a detectable aggregate gain. Variants that adapt latent context or selected parameters provide mixed results, suggesting that latent‑prediction objectives should be evaluated via closed‑loop rollouts rather than latent loss alone.
arXiv:2610.27851v1 Announce Type: cross Abstract: This paper shows that JEPA-style latent world models (LWMs) can learn across diverse domains and data sources by adopting the language-conditioned in...
arXiv:2610.28122v1 Announce Type: cross Abstract: Cascade world-action models (WAMs) use predicted visual futures to condition robot actions, but inherit costly iterative inference from diffusion-pre...
La‑JEPA introduces a compact world‑action model that trains a policy transformer from scratch in the feature space of a pretrained V‑JEPA encoder, learning to predict future features jointly with actions during training but generating only actions at inference. The approach achieves state‑of‑the‑art performance on RoboTwin 2.0 and LIBERO‑Plus while using far fewer parameters and running faster than comparable methods such as Fast‑WAM, LeapBot‑WA, and JEPA‑WAM. A design study shows that JEPA features outperform video VAE latents, especially out‑of‑distribution, and that training‑time future‑prediction choices significantly influence policy performance even when future prediction is omitted at inference.
arXiv:2610.09940v1 Announce Type: cross Abstract: Joint-embedding predictive architectures (JEPAs) predict masked or future observations in representation space, offering a natural source of predicti...
The paper introduces NAVA-WAM, a method that learns action priors directly from observation-only videos to pretrain action policies for robots. It uses a two-stage training process: first, it pretrains on videos with future‑video flow‑matching supervision to learn action‑relevant priors, and second, it fine‑tunes with action‑labeled demonstrations for robot control. Experiments show that NAVA‑WAM outperforms previous methods in both in‑distribution and out‑of‑distribution scenarios, achieving strong action‑label efficiency and real‑robot generalization.
The paper investigates how different visual representations and integration mechanisms affect the performance of World Action Models (WAMs). It finds that strong general‑purpose visual representations yield the largest robustness gains, that channel‑wise joint prediction is the most effective integration strategy, and that additional sources should be chosen based on the unique task‑relevant information they provide. Using these insights, the authors propose R$^2$‑WAM, which achieves state‑of‑the‑art success rates on RoboCasa GR1 Tabletop and RoboTwin 2.0.
The paper introduces DRILL, a bilevel training framework that optimizes world models for policy improvement rather than pixel accuracy. DRILL-VWI reweights prediction errors using policy visitation and model confidence, while DRILL-IMG employs meta‑gradients to directly maximize downstream policy returns. Experiments on manipulation simulators show DRILL-IMG boosts success rates by 13.1 points over WMPO and 7.9 points over RLVR-World, demonstrating that world models can be more effectively trained using policy gradients.
JEPA-WAM enhances World Action Models (WAMs) by pairing text instructions with stochastically generated visual cues, using a text-to-image generator and a frozen V‑JEPA encoder to create dense goal representations. These representations are compressed into goal tokens that condition both video and action experts via cross‑attention, enabling the model to better ground instructions. On a new real‑robot benchmark, JEPA‑WAM attains 87.3%, 74.5%, and 80.9% success rates across in‑distribution, out‑of‑distribution scenes, and out‑of‑distribution instructions, outperforming prior methods by significant margins.