arXiv AI
Oct 15

OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism

OxyGen introduces a unified KV cache management system for Vision‑Language‑Action (VLA) models that run multiple tasks in parallel, such as manipulation, conversation, and memory construction. By treating the KV cache as a shared resource, it eliminates redundant prefill operations and enables cross‑frame continuous batching, leading to significant speedups—up to 3.7× faster than isolated execution—while maintaining high language throughput and action frequency on both RTX 4090 and Jetson AGX Thor platforms. The approach is validated across several backbones and expert configurations, including deployment on a humanoid robot with on‑board Jetson hardware.

By Xiangyu Li, Huaizhi Tang, Xin Ding, Weijun Wang, Ting Cao, Yunxin Liu
arXiv AI
Aug 18

Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification

arXiv:2608. 15636v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in the field of embodied AI, but their high computational cost and limited predicted action length hinder real-time deployment.

By Chunyu Qi, Zhuoran Song, Jian Weng, Haozhe Jiang, Xueyuan Liu, Naifeng Jing, Guanghui He, Xiaoyao Liang, Haibing Guan