arXiv AI

VLANeXt: Recipes for Building Strong VLA Models

arXiv AI
3d ago

VLANeXt Family: A Systematic Study of VLA Models from Core Recipes to Emerging Paradigms

The paper presents VLANeXt, a Vision‑Language‑Action (VLA) model built from a unified framework that systematically analyzes design choices across foundational components, perception essentials, and action modeling. By dissecting these dimensions, the authors identify 12 key findings that form a practical recipe for strong VLA models, and demonstrate that VLANeXt outperforms state‑of‑the‑art methods on LIBERO and LIBERO‑plus benchmarks while also excelling in real‑world experiments. The study further extends VLANeXt into a family of variants—scaled models, latent‑action pretraining, latent predictive representation learning, and world action modeling—showing that the core recipe remains effective across different scales and emerging paradigms.

By Xiao-Ming Wu, Kang Liao, Yihang Luo, Bin Fan, Jian-Jian Jiang, Runze Yang, Zhonghua Wu, Wei-Shi Zheng, Chen Change Loy
arXiv AI
Jun 2

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.

By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen
arXiv AI
Sep 18

StarVLA-$\alpha$: Reducing Complexity in Vision-Language-Action Systems

StarVLA-α is a streamlined Vision‑Language‑Action (VLA) model that reduces architectural and pipeline complexity to facilitate systematic study of VLA design choices. By employing a strong VLM backbone and minimal design, it achieves competitive performance across multiple benchmarks (LIBERO, SimplerEnv, RoboTwin, RoboCasa) and outperforms the baseline π₀.₅ by 20% on the RoboChallenge benchmark. The authors plan to release the code to support future VLA research.

By Jinhui Ye, Ning Gao, Senqiao Yang, Jinliang Zheng, Zixuan Wang, Yuxin Chen, Pengguang Chen, Yilun Chen, Shu Liu, Jiaya Jia
arXiv Machine Learning
Jun 18

Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models

arXiv:2606. 19297v1 Announce Type: new Abstract: Embodied Vision-Language-Action (VLA) models are typically obtained by fine-tuning powerful pretrained VLMs on robotics data, yet it is unclear how much commonsense and factual knowledge they retain after adaptation.

By Nikita Kachaev, Andrey Moskalenko, Matvey Skripkin, Nikita Kurlaev, Daria Pugacheva, Albina Burlova, Mikhail Kolosov, Denis Shepelev, Andrey Kuznetsov, Elena Tutubalina, Aleksandr I. Panov, Alexey K. Kovalev, Vlad Shakhuro
Hugging Face Trending Papers
Jun 17

Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models

Embodied Vision-Language-Action (VLA) models are typically obtained by fine-tuning powerful pretrained VLMs on robotics data, yet it is unclear how much commonsense and factual knowledge they retain after adaptation. Failures on knowledge-sensitive tasks are ambiguous, conflating missing knowledge with poor generalization of low-level control.

arXiv AI
Jul 13

CLAP: Direct VLM-to-VLA Adaptation via Language-Action Grounding

arXiv:2607. 08974v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) inherit semantic capabilities from pretrained VLMs, yet large-scale post-training on robot data and architectural modifications can reshape the backbone so extensively that it becomes difficult to isolate what the VLM contributes to control.

By Yuri Ishitoya, Jeremy Siburian, Masashi Hamaya, Kuniaki Saito, Cristian C. Beltran-Hernandez, Mai Nishimura
arXiv Machine Learning
Jun 11

Learning What to Say to Your VLA: Mostly Harmless Vision Language Action Model Steering

arXiv:2606. 12299v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models provide a natural language interface to robot control, but the mapping from language to behavior is often brittle and unintuitive: semantically similar instructions can induce drastically different behaviors, while some capabilities may not be elicitable through prompting alone.

By Hyun Joe Jeong, Gokul Swamy, Andrea Bajcsy
arXiv Machine Learning
Jul 7

WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling

arXiv:2607. 03461v1 Announce Type: cross Abstract: World models aim to capture environment dynamics in ways that support perception, reasoning, and action, and have recently become a central direction in Vision-Language-Action-World (VLAW) modeling.

By Zelin Zhao, Min Shi, Bo Yuan, Haotian Xue, Jialuo Li, Lama Moukheiber, Humphrey Shi, Yongxin Chen
arXiv Computer Vision
Sep 4

VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

VKnowU is a benchmark that tests multimodal large language models (MLLMs) on their grasp of visual knowledge—intuitive, human-like understanding of physical and social principles in videos. The benchmark contains 1,680 questions across 1,249 videos, covering eight core types of visual knowledge, and shows that current state‑of‑the‑art MLLMs still lag behind human performance, especially on world‑centric tasks. To address this gap, the authors release VKnowQA and VideoKnow+, a baseline model that incorporates visual knowledge via a See‑Think‑Answer framework and reinforcement learning, improving performance on VKnowU and related datasets.

By Tianxiang Jiang, Sheng Xia, Yicheng Xu, Linquan Wu, Xiangyu Zeng, Limin Wang, Yu Qiao, Yi Wang