arXiv Computer Vision
Vision Token Pruning via Query-Vision Interaction Decomposition
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Sep 30
TReVS: Integrating Textual Relevance and Visual Saliency for Efficient Vision-Language Model Token Pruning
arXiv:2609.37581v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) excel at visual understanding and reasoning but often incur substantial inference costs due to the large number of visu...
arXiv Computer Vision
Oct 19
Query-adaptive Recurrent Tokenization
The paper introduces Query-Adaptive Recurrent Tokenizer (QART), a backbone-agnostic module that tailors visual representations to a text query and dynamically adjusts the token budget. QART starts from frozen encoder features and iteratively adds tokens until the query-relevant visual information is captured with sufficient fidelity, guided by reconstruction quality. Experiments on Visual7W, VQAv2, CORE, and PODS show that QART achieves high accuracy while using only about 29 visual tokens, a 20–25× reduction compared to dense vision-language models, and outperforms existing methods such as SteerViT on certain benchmarks.
arXiv Computer Vision
Aug 27
Not All Attention Heads Contribute to Critical Visual Token Selection: Head-Aware Pruning Matters More
The paper shows that only a small subset of attention heads in vision-language models is responsible for selecting critical visual tokens. By pruning tokens based on similarity before LLM reasoning and then applying head‑aware pruning during reasoning, the proposed ProViP framework achieves high task performance with significant speedups. Experiments on LLaVA‑1.5‑7B demonstrate 95.9% performance retention and a 1.62× inference speedup at an 88.9% pruning ratio.
arXiv AI
Jun 29
IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models
arXiv:2604. 00757v2 Announce Type: replace-cross Abstract: Large Vision Language Models show impressive performance across image and video understanding tasks, yet their computational cost grows rapidly with the number of visual tokens.
Hugging Face Trending Papers
Sep 28
When Text Matters: Design Principles for Visual Token Pruning in Vision-Language Model
The paper introduces a training‑free visual token pruning strategy for vision‑language models that separates early vision‑guided pruning from later text‑guided reselection. By first pruning tokens with vision‑encoder attention, retaining candidates until the decoder midpoint, and then applying text‑to‑visual attention, the method preserves task‑relevant visual information. Across eight benchmarks and three models, it achieves an average performance recovery of 11.10 and 16.84 percentage points at 80% and 90% pruning, respectively, while maintaining comparable or lower LLM‑prefill latency.
arXiv Computer Vision
Oct 9
LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models
LearnPruner rethinks token pruning in vision‑language models by addressing shortcomings of attention mechanisms in vision encoders and large language models. The method uses a two‑stage approach: first a learnable pruning module removes redundant vision tokens after the vision encoder, then only task‑relevant tokens are kept in the LLM’s middle layer. Experiments show that LearnPruner retains about 95% of performance while using only 5.5% of vision tokens, achieving a 3.2× speed‑up.