← Back to all news
arXiv Machine Learning By Ahad Jawaid, Yu Xiang

NAC: Neural Action Codec for Vision-Language-Action Models

Read the original on arXiv Machine Learning →

The paper introduces the Neural Action Codec (NAC), a convolutional encoder‑decoder architecture that treats short robot action trajectories as multi‑channel 1D signals and compresses them using a multi‑scale residual vector quantization (RVQGAN) model. NAC replaces traditional discrete action tokenizers with a compact, ordered token space via offset codebooks, allowing standard autoregressive policies to operate over short, structured sequences while a Vocos‑style decoder reconstructs the actions. Experiments on LIBERO‑10, RoboMimic, and real‑world manipulation tasks show that NAC achieves higher reconstruction fidelity and better average success rates than existing binning, FAST, and VQ‑based tokenizers at comparable or improved compression rates.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Sep 18

M2Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models

The paper introduces M2Tok, a Multi-head Multi-codebook Action Tokenizer that reduces reconstruction loss for continuous action signals by decomposing latent features into multiple heads and assigning independent codebooks to each. This design expands representational expressivity, leading to lower reconstruction error and higher success rates in Vision‑Language‑Action models evaluated on RoboTwin, Simpler‑Env, and zero‑shot real‑world tasks. Ablation studies confirm the effectiveness of both multi‑head and multi‑codebook mechanisms.

By Chunpu Xu, Zhixuan Liang, Yuhao Zhang, Chi-Min Chan, Jessie Wang, Yang Xiao, Mengkang Hu, Xiaokang Yang, Yao Mu
llmsnlpefficiencymultimodalbenchmarks
More like this →
arXiv AI
Sep 17

${M}^2$Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models

The paper introduces ${M}^2$Tok, a Multi-head Multi-codebook Action Tokenizer that reduces reconstruction loss in discrete action tokenization for Vision‑Language‑Action models. By decomposing latent action features into multiple heads and assigning independent codebooks to each, the tokenizer expands representational expressivity and improves policy performance. Experiments on RoboTwin, Simpler‑Env, and zero‑shot real‑world tasks show superior reconstruction fidelity and higher success rates compared to prior methods.

By Chunpu Xu, Zhixuan Liang, Yuhao Zhang, Chi-Min Chan, Jessie Wang, Yang Xiao, Mengkang Hu, Xiaokang Yang, Yao Mu
llmsnlpefficiencymultimodalbenchmarks
More like this →
arXiv AI
Jun 19

Finetuning Vision-Language-Action Models Requires Fewer Layers Than You Think

arXiv:2606. 20246v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation, yet their multi-billion parameter architectures impose prohibitive computational burdens during downstream fine-tuning and real-time inference.

By Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha, Khoa Vo, Philip Lund M{\o}ller, Quang T. Nguyen, Long Dinh, Tuan Dam, Vu Duong, Tung M. Luu, Trung Le, Tran Nguyen Le, Minh Vu, An Thai Le, Ngan Le, Daniel Sonntag, James Zou, Jan Peters, Duy M. H. Nguyen, Ngo Anh Vien
roboticsfine-tuningmultimodalbenchmarkssafety
More like this →