The paper introduces Varest, a method for image-token compression that reduces computational load while preserving semantic integrity. Varest uses variable-resolution tokenization guided by semantic fungibility, merging tokens within coherent regions coarsely and maintaining finer distinctions at object boundaries and text. After early decoder processing, learned relevance registers prune visual tokens, enabling a single checkpoint to perform well across multiple budgets and backbones, achieving 86.10‑92.33% of full-token performance with only 5.56% of the original token count.
arXiv:2610.17412v1 Announce Type: cross Abstract: Vision Transformers incur high computational costs when processing long visual token sequences. Existing reduction methods typically rank tokens inde...
Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive. Existing token compression methods reduce this cost, yet they can remove or collapse object- and region-level evidence that future query tokens may need to select.
ERASE is an adaptive two-stage token pruning framework designed to reduce computational overhead in Vision‑Language Models by eliminating redundant visual tokens. Stage 1 removes image‑level redundancy using lightweight raw‑image statistics, while Stage 2 progressively prunes instruction‑irrelevant tokens across decoder layers. Experiments on Qwen2.5‑VL‑7B show that ERASE retains 95.70 % of the original accuracy while keeping only 25 % of the tokens.
The paper introduces a training‑free visual token pruning strategy for vision‑language models that separates early vision‑guided pruning from later text‑guided reselection. By first pruning tokens with vision‑encoder attention, retaining candidates until the decoder midpoint, and then applying text‑to‑visual attention, the method preserves task‑relevant visual information. Across eight benchmarks and three models, it achieves an average performance recovery of 11.10 and 16.84 percentage points at 80% and 90% pruning, respectively, while maintaining comparable or lower LLM‑prefill latency.
arXiv:2607. 04605v1 Announce Type: cross Abstract: Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive.
arXiv:2609.24485v1 Announce Type: new Abstract: Visual token pruning is a promising approach to reducing the inference cost of large vision-language models (LVLMs), yet aggressive token reduction oft...
arXiv:2609.05916v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose subs...
The paper introduces ReFIT, an instruction‑guided visual token reduction framework designed to accelerate large vision‑language model inference. ReFIT combines Relevance‑Guided Window Reshaping (RWR) to adaptively capture instruction‑relevant regions and Instruction‑Guided Token Refinement (ITR) to prune unnecessary visual tokens. Experiments on four VQA benchmarks show that ReFIT improves answer accuracy while lowering computational cost, and qualitative results confirm its ability to localize relevant regions and remove extraneous visual information.
The paper shows that only a small subset of attention heads in vision-language models is responsible for selecting critical visual tokens. By pruning tokens based on similarity before LLM reasoning and then applying head‑aware pruning during reasoning, the proposed ProViP framework achieves high task performance with significant speedups. Experiments on LLaVA‑1.5‑7B demonstrate 95.9% performance retention and a 1.62× inference speedup at an 88.9% pruning ratio.
The paper introduces Adaptive Visual Token Pruning (AVTP), a training‑free framework that dynamically selects pruning layers and ratios for large vision‑language models (LVLMs) when processing multiple image sequences. By analyzing visual attention distributions across different LVLM architectures, AVTP adapts token retention to image importance, enabling efficient inference without relying on attention‑based computations incompatible with FlashAttention. Experiments show significant speedups—up to 2× for Qwen3VL‑8B—while preserving or even improving accuracy on multi‑image benchmarks.
LearnPruner rethinks token pruning in vision‑language models by addressing shortcomings of attention mechanisms in vision encoders and large language models. The method uses a two‑stage approach: first a learnable pruning module removes redundant vision tokens after the vision encoder, then only task‑relevant tokens are kept in the LLM’s middle layer. Experiments show that LearnPruner retains about 95% of performance while using only 5.5% of vision tokens, achieving a 3.2× speed‑up.
Visual token pruning is a promising approach to reducing the inference cost of large vision-language models (LVLMs), yet aggressive token reduction often causes substantial performance degradation. We...