arXiv AI
Oct 2

VETO: Video Efficient Token Optimization for Vision Language Models

VETO (Video Efficient Token Optimization for Vision Language Models) is a plug‑in that reduces the quadratic cost of visual tokens in long‑video inference by applying dual‑axis compression: an intra‑frame compressor merges semantically similar tokens within each frame, and an inter‑frame compressor merges temporally redundant frames. By first compressing spatial dimensions, VETO lowers the cost of subsequent global temporal matching, surpassing single‑axis methods and achieving up to 45% faster inference on models such as LLaVA‑OneVision‑7B while maintaining or improving accuracy. The approach is universally applicable across LLaVA‑OneVision, InternVL‑2.5, and LongVA, preserving or enhancing zero‑shot accuracy even under extreme token budgets.

By Gueter Josmy Faure, Hao Ping Wang, Min-Hung Chen, Winston H. Hsu
arXiv Computer Vision
Oct 28

Training-Free Inference Acceleration for Video Large Language Models: Methods, Trade-offs, and Evaluation

The paper surveys training‑free methods to accelerate inference in Video Large Language Models (VLMs), aiming to reduce computational, latency, and memory costs while retaining task‑relevant spatiotemporal evidence. It categorizes techniques by their intervention points in the inference pipeline—input, visual encoding, projector, prefill, and decoding—and discusses post‑training quantization as a cross‑stage approach. The authors analyze how the position of each intervention influences evidence preservation and computational savings, noting that local efficiency gains may not lead to overall acceleration, and they provide quantitative comparisons from prior work and new experiments to evaluate accuracy‑latency trade‑offs.

By Yuqi Li, Chuanyue Yan, Neil Rodrigues, Xianglong Wang, Xiaoqin Feng, Junming Liu, Xi Xiao, Zhongwei Cheng, Wei Li, Lin Chen, Yingli Tian
arXiv Computer Vision
Nov 9

AdaAlloc: Adaptive Visual Token Allocation for Long-Video Question Answering

AdaAlloc is a training‑free inference framework that adaptively allocates visual tokens between a global video overview and localized visual detail for long‑video question answering. It first plans a global/local allocation policy, uses temporal grounding to locate candidate evidence segments, and refines them into a compact temporal memory. The selected policy controls both frame sampling density and resolution, combining sparse, lower‑resolution global frames with denser, higher‑resolution local frames, and achieves up to 10.5 percentage‑point improvements over budget‑matched uniform sampling on several benchmarks.

By Joungbin An, Kristen Grauman
arXiv Computer Vision
Oct 12

Video Codec-Guided Training-Free Token Compression for Video Large Language Models

Video large language models (VLLMs) need long visual token sequences, making token compression essential for efficient inference. This paper introduces a training‑free token compression method that leverages existing video codec information—such as reference‑frame dependencies, motion vectors, and residuals—to guide low‑cost search for token protection and temporal merging. Experiments on three VLLMs and four benchmarks show that the method matches or surpasses the FlashVID baseline while dramatically reducing feature comparisons.

By Xihua Sheng, Ruijie Liu, Chang Wen Chen