arXiv Computer Vision

From Pixels to Actions - Towards Native Vision-Language-Action Policies

The paper introduces NEO-VLA, an encoder‑free vision‑language‑action framework that tokenizes images with lightweight patch convolutions and lets visual and language features evolve together throughout the backbone. By allowing a flow‑matching action expert to access instruction‑conditioned visual representations at multiple depths, NEO‑VLA consistently outperforms encoder‑based counterparts across diverse manipulation tasks and distribution shifts. Layer‑wise probing reveals that action‑critical signals peak early in the visual hierarchy, underscoring the benefit of accessing representations across the computational pipeline rather than relying on a fixed encoder output.

arXiv AI
Sep 10

LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policies

arXiv:2609.06079v1 Announce Type: new Abstract: Vision-Language-Action (VLA) policies leverage pretrained vision-language models (VLMs) to guide action generation for robot control. VLMs provide hier...

By Zheng Lu, Haoran Liao, Wanqi Zhong, Yunhe Ni, Lijie Wang, Xingjie Fan, Zhisheng Chen, Yantang Qu, Meijia Chen, Tianyu Xin, Zirui Song, Yiming Li
arXiv Computer Vision
Aug 27

V-Link: Recovering Lost Visual Representations in Action DiT for Vision-Language-Action Models

V-Link is a method designed to enhance Vision‑Language‑Action (VLA) models by recovering visual representations during the transfer from vision‑language (VL) features to action (A) features. It introduces complementary Spatial and Semantic Query representations that are injected into Action DiT through asymmetric pathways, providing both semantic augmentation and dedicated geometric conditioning for action generation. Experiments on LIBERO, LIBERO‑Plus, RoboTwin 2.0, and real‑world AGIBOT A3 Ultra tasks show significant performance gains over the base GR00T N1.6 model.

By Yehao Lu, Jiarui Yang, Yuning Su, Yufeng Xie, Yu Zhong, Yazhou Zhang, Haiyu Lan, Kaixiang Lu, Peiwen Lin, Chuang Wang, Zequn Qin, Enyu Li, Xi Li
arXiv AI
Jun 2

VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.

By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen
arXiv AI
Jun 19

Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think

arXiv:2606. 20246v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation, yet their multi-billion parameter architectures impose prohibitive computational burdens during downstream fine-tuning and real-time inference.

By Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha, Khoa Vo, Philip Lund M{\o}ller, Quang T. Nguyen, Long Dinh, Tuan Dam, Vu Duong, Tung M. Luu, Trung Le, Tran Nguyen Le, Minh Vu, An Thai Le, Ngan Le, Daniel Sonntag, James Zou, Jan Peters, Duy M. H. Nguyen, Ngo Anh Vien
arXiv Computer Vision
Oct 1

Correcting WHERE, Preserving HOW: Compositional Generalization for Vision-Language-Action Models via Referential Guidance

arXiv:2609.38616v1 Announce Type: cross Abstract: While Vision-Language-Action (VLA) models enable flexible action generation, their generalization across diverse environmental elements, including ma...

By Yanyan Zhang, Disheng Liu, Xinpeng Li, Chaoda Song, Mohsen Hariri, Debargha Ganguly, Wang Yang, Kai Ye, Bryce Grant, Vipin Chaudhary, Yu Yin
arXiv Machine Learning
Sep 22

Beyond Appearance Shifts: Task-Semantic Action Calibration for VLA Models

The paper introduces BAS‑VLA, a task‑semantic action calibration framework for vision‑language‑action models that addresses two failure modes: unnecessary action drift under appearance changes and insufficient behavioral change under semantic alterations. BAS‑VLA uses a breaking‑centered calibration core and a selective evidence‑gated preserving auxiliary to maintain performance on clean and semantics‑preserving conditions while suppressing stale‑task behavior. Experiments on OpenPI‑pi0.5 and LIBERO‑Object Milk‑Swap show high success rates on clean and preserved tasks, a dramatic drop under target‑object swaps, and improved robustness to style shifts from 42% to 70% without harming clean performance.

By Shuaijun Liu, Feiyang You, Chengyu Wu, Shuyang Hao, Chenglong Zhang, Jingyao Cai, Xingwei Chen, Ningxin Su
arXiv Computer Vision
Sep 30

RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation

RawVLA introduces a streaming neural image signal processor that adaptively renders RAW observations for vision‑language‑action (VLA) policies, focusing on imaging factors that influence embodied behavior. The authors systematically analyze how five ISP dimensions—gain, sensor noise, chromatic response, tonal response, and bit depth—affect action prediction and manipulation success, showing that RAW‑to‑RGB processing significantly shapes outcomes. They also present RawVLA‑Bench, a RAW‑domain manipulation benchmark that evaluates image processing as an explicit variable across clean and adverse conditions, demonstrating that RawVLA maintains performance under standard settings while markedly improving robustness under degraded imaging.

By Shuhong Liu, Heng Zhou, Lingfeng Qian, Yuhao Fang, Xianbao Hou, Qianyu Zhou, Lin Gu, Wei Sui, Jianfei Yang, Ziteng Cui
arXiv AI
Jun 16

Action with Visual Primitives

arXiv:2605. 22183v3 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models have emerged as a promising paradigm for generalist robotic manipulation.

By Weilong Guo, Yuchen Wang, Renping Zhou, Yunfeng Zhang, Rui Fang, Yuyang Pang, Wenda Xu, Gao Huang
arXiv Computer Vision
Oct 2

DynamicVLA: A Vision-Language-Action Model for Dynamic Object Manipulation

DynamicVLA is a latency‑aware Vision‑Language‑Action model designed for dynamic object manipulation, featuring a compact 0.4B architecture and a convolutional vision encoder for efficient multimodal inference. It employs a continuous inference schedule that overlaps reasoning and execution, and a Latent‑aware Action Streaming mechanism that discards stale action prefixes to maintain action‑time alignment. The authors also introduce the Dynamic Object Manipulation (DOM) benchmark, comprising 200K synthetic episodes and 2K real‑world episodes, and demonstrate that DynamicVLA improves dynamic manipulation success in simulation and on real robots.

By Haozhe Xie, Beichen Wen, Jiarui Zheng, Zhaoxi Chen, Fangzhou Hong, Haiwen Diao, Ziwei Liu
arXiv AI
4d ago

STAR: Steering Action Representations during Inference

The paper introduces STAR (Steering Action Representations), a training‑free inference technique for vision‑language‑action models that addresses a late‑layer grounding gap. STAR applies an exponential moving average over late‑layer residual updates, enabling later action representations to inherit object‑aware update directions from earlier layers without changing the model. By incorporating recent proprioceptive states to adjust mixing strength, STAR improves object awareness in late layers and boosts average task success rates by 4.0 percentage points on robot manipulation benchmarks.

By Guangyu Zhao, Yongchang Zhang, Haoxuan Ru, Kaichen He, Haoran Wen, Kun Zhan, Zeyu Zhang, Yin Li, Yiwu Zhong