arXiv AI
StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 28
GOTS: Greedy Orthogonal Token Selection for High-Resolution Vision-Language Models
arXiv:2607. 23913v1 Announce Type: new Abstract: Modern vision-language models (VLMs) increasingly rely on dynamic or high-resolution visual encoding, producing thousands of visual tokens that substantially increase downstream language-model inference cost.
arXiv AI
Jul 9
AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning
arXiv:2607. 07033v1 Announce Type: cross Abstract: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query.
arXiv Computer Vision
Oct 19
CatchUp: Depth-Aware Re-Admission of Visual Tokens for Efficient Large Vision-Language Models
CatchUp is a training‑free framework that improves efficiency in large vision‑language models by depth‑aware re‑admission of visual tokens. It repeatedly evaluates query‑conditioned relevance, processes only selected tokens per decoder interval, and preserves deferred tokens for later re‑admission, using reference‑to‑current transition signals to approximate missed representation evolution. Experiments on three referring‑expression grounding datasets and six multimodal benchmarks show that CatchUp retains over 92% of full‑model performance with 88.9% fewer active tokens and reduces end‑to‑end latency by 8.0% at a 64‑token budget.
arXiv AI
Oct 15
When 1+1 Backfires: Rethinking Relevance-Diversity Balancing for Sample-Aware Visual Token Reduction
arXiv:2610.16615v1 Announce Type: new Abstract: Visual token compression has emerged as a critical technique for reducing the inference cost of Vision-Language Models. Existing methods select tokens...
arXiv Computer Vision
Sep 18
QCPruner: Query-Conditioned Population Coverage for Visual Token Pruning
QCPruner is a training‑free visual token pruning method that conditions both token selection and representation on the query via bilateral utility weighting. It fuses keyword‑matched query anchors with cross‑modal cues to compute a nonnegative facility‑location objective that is monotone and submodular, guaranteeing a (1‑1/e) greedy approximation. Across multiple multimodal large language models, QCPruner consistently outperforms existing pruning methods, achieving over 96% of unpruned performance even with very few tokens retained.
arXiv Computer Vision
Oct 1
MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference
arXiv:2609.34330v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated impressive performance in multimodal understanding, but processing large numbers of visu...