Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive. Existing token compression methods reduce this cost, yet they can remove or collapse object- and region-level evidence that future query tokens may need to select.
arXiv:2607. 04605v1 Announce Type: cross Abstract: Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive.
arXiv:2607. 07033v1 Announce Type: cross Abstract: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query.
arXiv:2609.36916v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) incur high inference latency from long visual token sequences. Existing pruning methods commonly use attention...
arXiv:2609.34330v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated impressive performance in multimodal understanding, but processing large numbers of visu...
The paper introduces PAYN, a training‑free token compression strategy for Multimodal Large Language Model (MLLM) based Referring Expression Segmentation (RES). By preserving original position embeddings and local spatial structures, PAYN retains tokens that are evenly distributed across neighboring regions, thereby maintaining spatial relational consistency. Experiments on multiple RES benchmarks show that PAYN outperforms existing token compression methods, confirming that position information alone is sufficient for effective compression in this task.
arXiv:2609.16841v1 Announce Type: cross Abstract: Increasing image resolution produces ever-longer visual-token sequences in vision-language models (VLMs), substantially raising their inference cost....
arXiv:2607. 23913v1 Announce Type: new Abstract: Modern vision-language models (VLMs) increasingly rely on dynamic or high-resolution visual encoding, producing thousands of visual tokens that substantially increase downstream language-model inference cost.
arXiv:2609.05916v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose subs...
The paper shows that only a small subset of attention heads in vision-language models is responsible for selecting critical visual tokens. By pruning tokens based on similarity before LLM reasoning and then applying head‑aware pruning during reasoning, the proposed ProViP framework achieves high task performance with significant speedups. Experiments on LLaVA‑1.5‑7B demonstrate 95.9% performance retention and a 1.62× inference speedup at an 88.9% pruning ratio.
arXiv:2610.01785v1 Announce Type: cross Abstract: Processing long videos with Vision-Language Models (VLMs) is bottlenecked by the quadratic cost of visual tokens, making long-form inference prohibit...
Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency.
The paper argues that token importance alone is insufficient to determine safe removal of visual tokens in multimodal large language models, because removability depends on representation depth and the surrounding deletion set. Through controlled experiments, the authors show that the same tokens can have different effects when removed at different depths or contexts. They introduce CoRePrune, a training‑free two‑stage pruning framework that refreshes deletion effects as visual representations evolve and refines candidate tokens based on the current deletion set, achieving high performance retention across multiple backbones and reducing prefill time significantly.