Hugging Face Trending Papers
LearnPrunerV2: Learnable Progressive Token Pruning for Vision-Language Models
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
2d ago
LearnPrunerV2: Learnable Progressive Token Pruning for Vision-Language Models
LearnPrunerV2 is a fully learnable token‑pruning framework for Vision‑Language Models that replaces heuristic pruning with lightweight modules in every backbone layer. These modules estimate token importance by combining visual saliency and query relevance via a shadow token, revealing a consistent staircase redundancy pattern across layers. The method introduces Budget‑Deterministic Pruning to set fixed per‑layer budgets, achieving state‑of‑the‑art efficiency while preserving 99.8% of performance even after pruning 77.8% of visual tokens on LLaVA‑v1.5.
arXiv Computer Vision
Oct 9
LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models
LearnPruner rethinks token pruning in vision‑language models by addressing shortcomings of attention mechanisms in vision encoders and large language models. The method uses a two‑stage approach: first a learnable pruning module removes redundant vision tokens after the vision encoder, then only task‑relevant tokens are kept in the LLM’s middle layer. Experiments show that LearnPruner retains about 95% of performance while using only 5.5% of vision tokens, achieving a 3.2× speed‑up.
arXiv Computer Vision
Aug 27
Not All Attention Heads Contribute to Critical Visual Token Selection: Head-Aware Pruning Matters More
The paper shows that only a small subset of attention heads in vision-language models is responsible for selecting critical visual tokens. By pruning tokens based on similarity before LLM reasoning and then applying head‑aware pruning during reasoning, the proposed ProViP framework achieves high task performance with significant speedups. Experiments on LLaVA‑1.5‑7B demonstrate 95.9% performance retention and a 1.62× inference speedup at an 88.9% pruning ratio.
arXiv AI
Sep 10
STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models
arXiv:2609.05916v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose subs...
arXiv Computer Vision
Sep 10
Beyond One-Size-Fits-All: Sample-Adaptive Strategy Routing for Vision Token Pruning in MLLMs
arXiv:2609.10346v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) process hundreds or thousands of visual tokens per image, incurring prohibitive inference costs. While existin...
arXiv AI
Sep 2
SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models
SinkPruner is a training‑free framework that prunes visual tokens for multimodal large language models by first removing high‑norm redundant tokens with a visual sanitizer and then selectively keeping tokens that align with the text query using a text‑guided pruner. The coarse‑to‑fine design reduces attention sink and dispersion, enabling an 89% token reduction while preserving 96.5% of LLaVA‑1.5’s performance and 91.8% of Qwen2.5‑VL’s performance across twelve image‑language and four video‑language benchmarks. The visual sanitizer also improves existing pruning methods, showing strong transferability.