ASAP (Attention Sink Anchored Pruning) is a training‑free framework designed to reduce the computational burden of Vision Transformers by addressing the attention sink problem. It models ViT information flow as a Lazy Random Walk, identifies the sink as a dominant probability accumulator, and uses diffusion distance to partition tokens via Radial Diffusion Clustering. The method then compresses background redundancy through Transition Weight Pooling, achieving up to 48% throughput acceleration while maintaining or improving accuracy across image, video, and vision‑language tasks.
arXiv:2509.22650v3 Announce Type: replace Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models,...
LearnPruner rethinks token pruning in vision‑language models by addressing shortcomings of attention mechanisms in vision encoders and large language models. The method uses a two‑stage approach: first a learnable pruning module removes redundant vision tokens after the vision encoder, then only task‑relevant tokens are kept in the LLM’s middle layer. Experiments show that LearnPruner retains about 95% of performance while using only 5.5% of vision tokens, achieving a 3.2× speed‑up.
The paper introduces STD, a hierarchical token pruning framework for Large Vision‑Language Models that aligns pruning strategies with the functional roles of different network stages. By using high‑frequency spectral analysis in shallow layers, Gaussian‑smoothed attention in intermediate layers, and a stability‑adaptive trigger in deep layers, STD preserves essential visual information while aggressively reducing token counts. Experiments demonstrate that STD outperforms existing pruning methods, achieving up to 94.4% token reduction and a 3.9× speed‑up on LLaVA‑NeXT‑7B.
The paper shows that only a small subset of attention heads in vision-language models is responsible for selecting critical visual tokens. By pruning tokens based on similarity before LLM reasoning and then applying head‑aware pruning during reasoning, the proposed ProViP framework achieves high task performance with significant speedups. Experiments on LLaVA‑1.5‑7B demonstrate 95.9% performance retention and a 1.62× inference speedup at an 88.9% pruning ratio.
The paper introduces V-CoLA, a training‑free token compression framework tailored for vision-language models that use linear attention. V-CoLA employs a uniqueness‑aware importance criterion to select critical vision tokens and an adaptive token merging strategy to compress them, all optimized to fit the chunk‑wise parallelism of linear attention. Experiments show that V‑CoLA retains 99.5% of the original performance with only half the vision tokens and achieves significant speedups during prefill.
arXiv:2606. 08156v1 Announce Type: cross Abstract: Vision Transformers (ViTs) achieve strong performance but suffer from high computational costs due to quadratic self-attention complexity.
arXiv:2606. 03569v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have demonstrated remarkable capabilities but suffer from significant computational overhead during inference.
arXiv:2609.05916v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose subs...
arXiv:2604. 00757v2 Announce Type: replace-cross Abstract: Large Vision Language Models show impressive performance across image and video understanding tasks, yet their computational cost grows rapidly with the number of visual tokens.
LearnPrunerV2 is a fully learnable token‑pruning framework for Vision‑Language Models that replaces heuristic pruning with lightweight modules in every backbone layer. These modules estimate token importance by combining visual saliency and query relevance via a shadow token, revealing a consistent staircase redundancy pattern across layers. The method introduces Budget‑Deterministic Pruning to set fixed per‑layer budgets, achieving state‑of‑the‑art efficiency while preserving 99.8% of performance even after pruning 77.8% of visual tokens on LLaVA‑v1.5.
arXiv:2606. 01503v1 Announce Type: cross Abstract: Unified vision-language models (VLMs) integrate visual understanding and visual generation within a single autoregressive backbone, but their joint training is computationally expensive and largely overlooked from an efficiency perspective.
The paper introduces Prune-and-Merge ViT, a training‑free token compression framework for Vision Transformers that combines class‑attention pruning with similarity‑based merging to reduce computational cost while preserving semantic information. It evaluates robustness using two new metrics—Relative Accuracy Drop (RAD) and Prediction Flip Rate (PFR)—and shows that the method maintains or improves accuracy on pretrained ViT and DeiT models compared to existing token reduction techniques. The authors also argue that the hybrid approach does not induce hallucination‑like behavior and keeps predictions stable under common image transformations.