arXiv AI
HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Computation and Language
Aug 28
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models
Visual Information-Guided Parallel Decoding for Diffusion Multimodal Large Language Models introduces the VIG‑Sampler, a method that prioritizes tokens for decoding based on their attention to image tokens and penalizes redundancy in image‑attention distributions. The approach aims to improve the quality of multimodal generation by selecting more informative tokens during diffusion decoding. Experiments on seven captioning and VQA benchmarks with three open‑source dMLLMs show that VIG‑Sampler outperforms the Info‑Gain Sampler by an average of 19.3 CIDEr points and achieves better COCO Caption results using only half as many decoding steps.
arXiv Machine Learning
Aug 6
DIVE: Dynamic Iterative Visual Evidence Construction for Efficient Vision-Language Models
arXiv:2608. 04496v1 Announce Type: cross Abstract: Visual inputs in vision-language models (VLMs) are often encoded into substantially longer token sequences than text, making visual tokens a major bottleneck for efficient inference.
arXiv AI
Sep 10
STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models
arXiv:2609.05916v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose subs...
Hugging Face Trending Papers
Aug 11
When Vision Becomes Text: Visual Token Pruning via Cross-Modal Residual Guidance in VLMs
Abundant visual information strengthens vision-language model (VLM) perception, yet massive visual tokens raise inference costs. Existing visual token pruning methods rely on similarity-based guidance, which exploits pairwise text-vision and vision-vision token correlations for compression.
arXiv Computation and Language
4d ago
SAGE: Sink-Aware Guided Emphasis for Visual Grounding in Vision-Language Decoders
The paper introduces SAGE, a lightweight method that guides decoder attention in vision-language models away from prompt‑invariant attention sinks (PIS) toward query‑dependent regions of interest (ROIs). By using token‑aligned ROI masks from vision backbones like CLIP, ViT, and DINOv3, SAGE improves visual grounding and reduces hallucinations across various VLM families. Experiments show consistent gains on public vision‑language benchmarks, especially in fine‑grained visual discrimination tasks where localized evidence is essential.
arXiv Computer Vision
Sep 10
Let ViT Speak: Generative Language-Image Pre-training
arXiv:2605.00809v3 Announce Type: replace Abstract: In this paper, we present \textbf{Gen}erative \textbf{L}anguage-\textbf{I}mage \textbf{P}re-training (GenLIP), a minimalist generative pretraining...