Hugging Face Trending Papers
Jul 27

MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning

Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency.

arXiv Computer Vision
Oct 19

CatchUp: Depth-Aware Re-Admission of Visual Tokens for Efficient Large Vision-Language Models

CatchUp is a training‑free framework that improves efficiency in large vision‑language models by depth‑aware re‑admission of visual tokens. It repeatedly evaluates query‑conditioned relevance, processes only selected tokens per decoder interval, and preserves deferred tokens for later re‑admission, using reference‑to‑current transition signals to approximate missed representation evolution. Experiments on three referring‑expression grounding datasets and six multimodal benchmarks show that CatchUp retains over 92% of full‑model performance with 88.9% fewer active tokens and reduces end‑to‑end latency by 8.0% at a 64‑token budget.

By Eunju Park, Uiwon Hwang