arXiv Computer Vision October 19, 2026 By Bavesh Balaji, Pranav Gopinath, Pouya Sadeghi, Sirisha Rambhatla

Query-adaptive Recurrent Tokenization

Read the original on arXiv Computer Vision →

The paper introduces Query-Adaptive Recurrent Tokenizer (QART), a backbone-agnostic module that tailors visual representations to a text query and dynamically adjusts the token budget. QART starts from frozen encoder features and iteratively adds tokens until the query-relevant visual information is captured with sufficient fidelity, guided by reconstruction quality. Experiments on Visual7W, VQAv2, CORE, and PODS show that QART achieves high accuracy while using only about 29 visual tokens, a 20–25× reduction compared to dense vision-language models, and outperforms existing methods such as SteerViT on certain benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Computer Vision

4d ago

CatchUp: Depth-Aware Re-Admission of Visual Tokens for Efficient Large Vision-Language Models

CatchUp is a training‑free framework that improves efficiency in large vision‑language models by depth‑aware re‑admission of visual tokens. It repeatedly evaluates query‑conditioned relevance, processes only selected tokens per decoder interval, and preserves deferred tokens for later re‑admission, using reference‑to‑current transition signals to approximate missed representation evolution. Experiments on three referring‑expression grounding datasets and six multimodal benchmarks show that CatchUp retains over 92% of full‑model performance with 88.9% fewer active tokens and reduces end‑to‑end latency by 8.0% at a 64‑token budget.

By Eunju Park, Uiwon Hwang

llms efficiency multimodal benchmarks

arXiv Machine Learning

Oct 8

ReToken: Improving Long-Context VLMs with Visual Retrieval Token

arXiv:2607.28627v2 Announce Type: replace-cross Abstract: Long visual contexts challenge vision-language models: performance degrades as the number of distractors grows, and processing all tokens at...

By Yao Xiao, Reuben Tan, Zhen Zhu, Yuqun Wu, Jianfeng Gao, Derek Hoiem

llms rag efficiency multimodal benchmarks

arXiv AI

Sep 16

StackTok: Accelerating VLMs Inference with Budget-Adaptive Visual Token Selection

arXiv:2609.16841v1 Announce Type: cross Abstract: Increasing image resolution produces ever-longer visual-token sequences in vision-language models (VLMs), substantially raising their inference cost....

By Zhenbin Wang, Lei Zhang, Lituan Wang, Wei Huang, Yan Wang, Zhenwei Zhang

llms multimodal benchmarks

arXiv Machine Learning

Jul 31

ReToken: One Token to Improve Vision-Language Models for Visual Retrieval

arXiv:2607. 28627v1 Announce Type: cross Abstract: Long visual context poses a challenge for vision-language models: performance degrades as the number of distractors grows, and processing all tokens at once is computationally infeasible under GPU memory constraints.

By Yao Xiao, Reuben Tan, Zhen Zhu, Yuqun Wu, Jianfeng Gao, Derek Hoiem

llms rag efficiency multimodal benchmarks

arXiv AI

Jul 9

AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning

arXiv:2607. 07033v1 Announce Type: cross Abstract: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query.

By Kyuan Oh, Bumsoo Kim

llms efficiency multimodal benchmarks

arXiv Computer Vision

Sep 22

PREM: Prefix-Steered Recurrent Memory for Long-Video Understanding

arXiv:2609.23601v1 Announce Type: new Abstract: Long-video understanding must capture transient visual evidence under strict token budgets, yet existing methods compress frames, append memory tokens,...

By Siru Zhong, Qiongyan Wang, Xiaohui Lv, Yuzheng Zhuang, Shuai Tao, Wulong Liu, Haohuan Fu, Yuxuan Liang