Diffusion-based vision-language-action (VLA) models often inherit the image-generation view: actions are generated by iterative denoising. We argue that VLA action generation has a different condition-target structure: the policy is conditioned on rich observations, language, and state, but predicts only a compact, low-dimensional action chunk.
arXiv:2606. 05737v1 Announce Type: cross Abstract: Diffusion-based vision-language-action (VLA) models often inherit the image-generation view: actions are generated by iterative denoising.
By Yitong Chen, Shiduo Zhang, Jingjing Gong, Xipeng Qiu
UniWALM is a unified world-action-language model that jointly learns video, action, and language in a single shared representation, using a mixture-of-transformers backbone with diffusion heads that co-denoise across modalities. By generating language as part of the same pass as action, the model closes a language‑as‑memory loop without an external planner, and introduces Flow Matching with Correction to maintain reliable action streams. Experiments show UniWALM outperforms baselines on compositional pick‑and‑place tasks and memory‑dependent benchmarks, and dramatically improves success on a real‑world Find‑the‑Item task when using generated textual memory.
By Rahul Sajnani, Diego Romeres, Mingyuan Zhang, Kai Gao, Carl Qi, Baichuan Huang, Srinath Sridhar, Dylan Randle, Lillian J. Ratliff, Jose Barreiros, Manikantan Nambi, Jeroen Vanbaar
TAPESTRY repurposes a pretrained video diffusion transformer into a robotic policy by treating robot actions as single-patch tokens in the model’s latent space, eliminating the need for video generation at inference. The lightweight linear projection maps each action timestep to the VAE patch dimensions, and the shared backbone is trained with three diffusion objectives—video diffusion, joint video+action diffusion, and action-only diffusion—to learn dynamics priors and action denoising. At test time, the model produces smooth action chunks in a single DDIM step, achieving strong performance on simulation and real‑robot benchmarks with significantly lower inference cost than comparable joint video–action policies.
By Jeremy A. Collins, Seungjae Lee, Rachanon Wachakorn, Krishnan Srinivasan, Animesh Garg, Vitor Campagnolo Guizilini, Paarth Shah
MT‑WAM enhances the Fast‑WAM framework by adding complementary supervision for future 2‑D point trajectories and visual features while keeping the original training objectives. A lightweight dual‑stream branch and structured attention mask isolate motion‑specific processing, and motion‑stream tokens provide additional dynamics cues to the action expert. During inference, MT‑WAM skips future‑video prediction, using cached video and motion information to achieve higher success rates on LIBERO, LIBERO‑Plus, RoboTwin 2.0 Clean2Rand, and several real‑world tasks.
By Yiguang Yang, Jiankun Peng, Xiaoming Wang, Yiran Zhang, Zhibo Fang
The paper introduces the Just Action Model (JAM), a framework that decouples backbone initialization, future observation objectives, and inference machinery to unify vision-language-action (VLA) and world-action model (WAM) approaches. JAM employs a pretrained visual backbone, an action expert, an action loss, and a foresight loss that predicts future encodings without visual generation, discarding the prediction head after training. Experiments show JAM outperforms FastWAM and ImageWAM on LIBERO, LIBERO-Plus, and RoboTwin, while reducing online training FLOPs by 73%.
By Yonghao Yu, Lang Huang, Runyi Li, Toshihiko Yamasaki