arXiv Machine Learning By Riccardo Andrea Izzo, Rimvydas Rubavicius, Gianluca Bardaro, Subramanian Ramamoorthy, Matteo Matteucci, Alessandro Suglia

Decoupled Early Exits for Task-Dependent Compute Allocation in Flow-Matching VLAs

Read the original on arXiv Machine Learning →

The paper introduces a framework for Flow‑Matching Vision‑Language‑Action (VLA) models that allows independent adjustment of backbone depth, action expert depth, and denoising steps. Lightweight Exit Transformers are added at intermediate layers to enable early exits, and a KV Cache synthesis mechanism manages skipped layers so the action expert can exit deeper than the backbone. Experiments on SmolVLA and π0.5 across LIBERO and Meta‑World show that joint tuning of these compute axes reduces latency by 79.2 % and FLOPs by 31.8 %, while improving mean success rate by 5.6 %.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Computer Vision
Oct 19

MambaAct: Efficient Mamba Action Experts for Flow-Matching Vision-Language-Action Models

MambaAct introduces a compact Mamba-based action expert to replace the transformer-based expert in flow‑matching vision‑language‑action models, preserving the pretrained visual‑language prefix encoder while reducing the action expert size by tenfold. The new design employs a lightweight Context Compressor to embed vision‑language conditioning into each Mamba block, eliminating repeated token‑level cross‑attention. Experiments show a 64.8 % reduction in denoising latency on LIBERO and 58.8 % in real‑world deployment, with a modest 1.08 % increase in mean success rate across multiple fine‑tuning seeds.

By Khuyen Pham, Tran Nguyen Le, Edoardo Fazzari, Cesare Stefanini
arXiv AI
Aug 13

G0.5: One Autoregressive Stream for Robot Reasoning and Action

arXiv:2608. 11739v1 Announce Type: cross Abstract: The prevailing recipe for Vision-Language-Action (VLA) models couples a pretrained VLM with a separately trained flow-matching action expert.

By Yicheng Liu, Zibin Dong, Baijun Ye, Tianyuan Yuan, Tao Jiang, Anqi Yang, Shicheng Cao, Haonan Liu, Yue Sun, Zihan Guo, Xiao Liu, Dong Ke, Changxun Pan, Chenru Wu, Tailai Cheng, Xiaoshu Ren, Xinlei Zhang, Jianning Cui, Zijie Zhao, Haoyu Zhang, Kaiming Xu, Haodong Yang, Bowen Zhang, Jiahui Niu, Shaoting Zhu, Shiduo Zhang, Hang Zhao
arXiv AI
Oct 28

TOPS: Reshaping Flow Source and Transport for Multi-Task Robot Manipulation

TOPS (Task‑Oriented Priors and Schedules) adapts flow‑based policies for language‑conditioned multi‑task robot manipulation by customizing the flow source prior and interpolation schedule per task without changing the network architecture. It constructs multimodal task sources from reverse‑mapped expert action sequences, allowing language instructions to select the generation start point, and adjusts the denoising schedule based on source‑target covariance mismatch to reduce velocity‑prediction risk. Experiments on LIBERO, LIBERO‑Plus, CALVIN, and real‑world tasks show significant performance gains, including a jump from 53.44 % to 69.67 % success on LIBERO‑10 with a single integration step and a five‑fold reduction in inference time.

By Tuo An, Chuhao Zhou, Jiahua Dong, Jingliang Li, Jiaqi Bai, Jindou Jia, Jianfei Yang
arXiv AI
Nov 6

Vision-Language Models Are Joint Denoisers in Robotic Action Generation

Vision‑Language Models Are Joint Denoisers in Robotic Action Generation introduces JADE, a Joint Action DEnoiser that lets a pretrained vision‑language model (VLM) and a diffusion action expert jointly refine actions. By structuring action latents, the VLM can read noisy actions alongside visual observations and language instructions, guiding the action expert through gated cross‑attention at each denoising step. This joint denoising reduces sampling steps, balances performance and latency, and improves success rates on RoboDojo and RoboTwin 2.0 compared to prior methods.

By Xicheng Gong, Wenxuan Song, Chengze Liu, Qiwei Li, Qize Yu, Jiaolong Yang, Haoang Li, Yadong Mu