arXiv Machine Learning

Beyond Reconstruction Error: Analytical and Data-Driven Action Tokenization for Autoregressive Vision-Language-Action Models

The paper investigates how different action tokenization methods affect closed‑loop control in autoregressive vision‑language‑action models. It compares analytical, linear, and nonlinear representations, showing that lower reconstruction error does not guarantee better policy performance. The study highlights the need to evaluate tokenization on multiple criteria, including sequence predictability and decoder stability, rather than relying solely on reconstruction fidelity.

arXiv AI
Sep 17

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

ActionPiece rethinks how actions are tokenized for autoregressive vision‑language‑action models by introducing physical rank consistency (PRC) to evaluate relational fidelity of reconstructed actions. The method jointly supervises representation learning and quantization to preserve local physical distance rankings, improving both PRC and policy success. Experiments on LIBERO, LIBERO‑Plus, SimplerEnv, and VLA‑Arena show significant gains over baseline tokenizers.

By Shijie Lian, Bin Yu, Zhaolong Shen, Xiaopeng Lin, Yichao Du, Zhirui Zhang, Laurence T. Yang, Kai Chen
arXiv AI
Sep 18

M2Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models

The paper introduces M2Tok, a Multi-head Multi-codebook Action Tokenizer that reduces reconstruction loss for continuous action signals by decomposing latent features into multiple heads and assigning independent codebooks to each. This design expands representational expressivity, leading to lower reconstruction error and higher success rates in Vision‑Language‑Action models evaluated on RoboTwin, Simpler‑Env, and zero‑shot real‑world tasks. Ablation studies confirm the effectiveness of both multi‑head and multi‑codebook mechanisms.

By Chunpu Xu, Zhixuan Liang, Yuhao Zhang, Chi-Min Chan, Jessie Wang, Yang Xiao, Mengkang Hu, Xiaokang Yang, Yao Mu
arXiv AI
Sep 17

${M}^2$Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models

The paper introduces ${M}^2$Tok, a Multi-head Multi-codebook Action Tokenizer that reduces reconstruction loss in discrete action tokenization for Vision‑Language‑Action models. By decomposing latent action features into multiple heads and assigning independent codebooks to each, the tokenizer expands representational expressivity and improves policy performance. Experiments on RoboTwin, Simpler‑Env, and zero‑shot real‑world tasks show superior reconstruction fidelity and higher success rates compared to prior methods.

By Chunpu Xu, Zhixuan Liang, Yuhao Zhang, Chi-Min Chan, Jessie Wang, Yang Xiao, Mengkang Hu, Xiaokang Yang, Yao Mu
arXiv AI
Oct 9

ActionCodec: What Makes for Good Action Tokenizers

The paper "ActionCodec: What Makes for Good Action Tokenizers" addresses the lack of design guidance for action tokenization in Vision‑Language‑Action (VLA) models. By deriving information‑theoretic principles—maximizing temporal token overlap, reducing vocabulary redundancy, boosting multimodal mutual information, and ensuring token independence—the authors propose ActionCodec, a tokenizer that improves training efficiency and VLA performance. Experiments show that a SmolVLM2‑2.2B fine‑tuned with ActionCodec reaches a 95.5 % success rate on LIBERO, rising to 97.4 % with architectural tweaks, setting a new state‑of‑the‑art for VLA models without robotics pre‑training.

By Zibin Dong, Yicheng Liu, Shiduo Zhang, Baijun Ye, Yifu Yuan, Fei Ni, Jingjing Gong, Xipeng Qiu, Hang Zhao, Yinchuan Li, Jianye Hao
arXiv AI
Aug 19

LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models

LoopVLA introduces a recurrent Vision‑Language‑Action architecture that learns to refine multimodal representations, predict actions, and estimate when further refinement is unnecessary. By iteratively applying a shared Transformer block and producing a sufficiency score at each step, it decouples refinement from fixed layer indices and aligns confidence scores with action quality through a self‑supervised objective. Experiments on LIBERO, LIBERO‑Plus, and VLA‑Arena demonstrate that LoopVLA reduces model parameters by 45% and boosts inference throughput up to 1.7× while matching or surpassing strong baselines in task success.

By Boyang Shen, Kaixiang Yang, Hao Wang, Qiuyu Yu, Qiang Xie, Qiang Li, Zhiwei Wang
arXiv Computer Vision
Oct 12

Dual-Stream Memory for Vision-Language-Action Policies

The paper introduces a dual‑stream memory architecture for vision‑language‑action policies, separating a task‑agnostic temporal compressor from a task‑conditioned visual retriever. The temporal compressor uses a Gated DeltaNet to condense history into a single token, while the retriever keeps frames above a relevance threshold and fuses them into spatial tokens. Evaluated on sixteen RoboMME tasks and four real‑robot tasks, the combined model outperforms the FrameSamp+Modul baseline, achieving higher mean success rates and notable gains on tasks requiring execution‑stage discrimination.

By Yang Wang, Ping Liu, Ruiyu Wang, Xuhua Chen, Lingfeng Zhang, Tao Zhang
arXiv AI
Jul 14

TS-Mask VLA: 2D Temporal-Spatial Masking for Vision-Language-Action Model with Effective Bridging

arXiv:2607. 09818v1 Announce Type: cross Abstract: Vision-language-action (VLA) models aim to understand natural-language instructions and visual observations, and to generate and execute corresponding actions as embodied agents.

By Shengzhuo Yang, Ronghao Yu, Chuanjie Lv, Linpeng Peng, Hang Yu, Jie Ren, Jiajun Lv, Yong Liu