arXiv Computer Vision
Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Jul 21
LookME: Lookup-Based Multimodal Embeddings for Layer Injection in Vision-Language Models
arXiv:2607. 16305v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) have achieved strong progress in multimodal understanding.
arXiv Computer Vision
Sep 10
Learning to Predict Middle-Layer Attention in MLLMs for Visual Token Pruning
arXiv:2608.06411v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) achieve strong performance across diverse vision-language tasks, but their efficiency is limited by...
arXiv AI
Sep 2
SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models
SinkPruner is a training‑free framework that prunes visual tokens for multimodal large language models by first removing high‑norm redundant tokens with a visual sanitizer and then selectively keeping tokens that align with the text query using a text‑guided pruner. The coarse‑to‑fine design reduces attention sink and dispersion, enabling an 89% token reduction while preserving 96.5% of LLaVA‑1.5’s performance and 91.8% of Qwen2.5‑VL’s performance across twelve image‑language and four video‑language benchmarks. The visual sanitizer also improves existing pruning methods, showing strong transferability.
arXiv AI
Aug 10
Learning to Predict Middle-Layer Attention in MLLMs for Visual Token Prunin
arXiv:2608. 06411v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) achieve strong performance across diverse vision-language tasks, but their efficiency is limited by the cost of processing numerous visual tokens.
arXiv Machine Learning
Jun 11
Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models
arXiv:2605. 25820v2 Announce Type: replace Abstract: Diffusion-based multimodal large language models (dMLLMs) decode by iteratively predicting tokens at multiple masked positions in parallel.
arXiv Computer Vision
Aug 27
Not All Attention Heads Contribute to Critical Visual Token Selection: Head-Aware Pruning Matters More
The paper shows that only a small subset of attention heads in vision-language models is responsible for selecting critical visual tokens. By pruning tokens based on similarity before LLM reasoning and then applying head‑aware pruning during reasoning, the proposed ProViP framework achieves high task performance with significant speedups. Experiments on LLaVA‑1.5‑7B demonstrate 95.9% performance retention and a 1.62× inference speedup at an 88.9% pruning ratio.