To adapt billion-parameter Vision Transformers for long-video understanding, recent methods freeze the backbone and train lightweight convolutional modules. While effective for parameter-efficient tra...
arXiv:2609.38984v1 Announce Type: cross
Abstract: World-action models (WAMs) leverage pretrained video models to improve generalization in robot control by jointly predicting future visual states and...
By Xinling Xie, Haodong Wang, Jiazhi Mi, Zhiming Liu, Zicong Hong, Xiaoyi Pang, Qianli Liu, Yangjia Hu, Ying Chen, Zhengyang Yan, Song Guo
arXiv:2606. 20246v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models pre-trained on massive video-robot datasets have revolutionized robotic manipulation, yet their multi-billion parameter architectures impose prohibitive computational burdens during downstream fine-tuning and real-time inference.
By Gia-Binh Nguyen, Trong-Bao Ho, Thien-Loc Ha, Khoa Vo, Philip Lund M{\o}ller, Quang T. Nguyen, Long Dinh, Tuan Dam, Vu Duong, Tung M. Luu, Trung Le, Tran Nguyen Le, Minh Vu, An Thai Le, Ngan Le, Daniel Sonntag, James Zou, Jan Peters, Duy M. H. Nguyen, Ngo Anh Vien
DynamicVLA is a latency‑aware Vision‑Language‑Action model designed for dynamic object manipulation, featuring a compact 0.4B architecture and a convolutional vision encoder for efficient multimodal inference. It employs a continuous inference schedule that overlaps reasoning and execution, and a Latent‑aware Action Streaming mechanism that discards stale action prefixes to maintain action‑time alignment. The authors also introduce the Dynamic Object Manipulation (DOM) benchmark, comprising 200K synthetic episodes and 2K real‑world episodes, and demonstrate that DynamicVLA improves dynamic manipulation success in simulation and on real robots.
By Haozhe Xie, Beichen Wen, Jiarui Zheng, Zhaoxi Chen, Fangzhou Hong, Haiwen Diao, Ziwei Liu
VideoTIR introduces a reinforcement‑learning approach to improve long‑video understanding by encouraging multimodal large language models to use comprehensive multi‑level toolkits efficiently. It combines Zero‑RL and SFT cold‑starting strategies to help models retrieve and focus on meaningful video segments, images, and regions, thereby reducing hallucinations. The method includes Toolkit Action Grouped Policy Optimization (TAGPO) to streamline tool‑calling and a sandbox‑based trajectory synthesis framework for high‑quality data, achieving strong results on three long‑video QA benchmarks.
By Zhe Gao, Shiyu Shen, Taifeng Chai, Weinong Wang, Haotian Xu, Xing Wu, Wenbin Li, Qi Fan, Yang Gao, Dacheng Tao
VETO (Video Efficient Token Optimization for Vision Language Models) is a plug‑in that reduces the quadratic cost of visual tokens in long‑video inference by applying dual‑axis compression: an intra‑frame compressor merges semantically similar tokens within each frame, and an inter‑frame compressor merges temporally redundant frames. By first compressing spatial dimensions, VETO lowers the cost of subsequent global temporal matching, surpassing single‑axis methods and achieving up to 45% faster inference on models such as LLaVA‑OneVision‑7B while maintaining or improving accuracy. The approach is universally applicable across LLaVA‑OneVision, InternVL‑2.5, and LongVA, preserving or enhancing zero‑shot accuracy even under extreme token budgets.
By Gueter Josmy Faure, Hao Ping Wang, Min-Hung Chen, Winston H. Hsu