arXiv Machine Learning July 31, 2026 By Yao Xiao, Reuben Tan, Zhen Zhu, Yuqun Wu, Jianfeng Gao, Derek Hoiem

ReToken: One Token to Improve Vision-Language Models for Visual Retrieval

Read the original on arXiv Machine Learning →

arXiv:2607. 28627v1 Announce Type: cross Abstract: Long visual context poses a challenge for vision-language models: performance degrades as the number of distractors grows, and processing all tokens at once is computationally infeasible under GPU memory constraints.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Jun 2

On the Limits of Token Reduction for Efficient Unified Vision Language Training

arXiv:2606. 01503v1 Announce Type: cross Abstract: Unified vision-language models (VLMs) integrate visual understanding and visual generation within a single autoregressive backbone, but their joint training is computationally expensive and largely overlooked from an efficiency perspective.

By Siyi Chen, Weiming Zhuang, Jingtao Li, Lingjuan Lv

llms multimodal

arXiv AI

Jun 2

StreamingVLM: Real-Time Understanding for Infinite Video Streams

arXiv:2510. 09608v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) could power real-time assistants and autonomous agents, but they face a critical challenge: understanding near-infinite video streams without escalating latency and memory usage.

By Ruyi Xu, Guangxuan Xiao, Yukang Chen, Liuning He, Yao Lu, Song Han

llms agents fine-tuning efficiency multimodal benchmarks safety

arXiv AI

Jun 11

AVIS: Adaptive Test-Time Scaling for Vision-Language Models

arXiv:2606. 11576v1 Announce Type: cross Abstract: Modern Vision-Language Models (VLMs) benefit from chain-of-thought prompting and test-time scaling, but these gains often come with prohibitive inference cost due to large visual contexts and long decoding chains.

By Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni, Hakki Can Karaimer, Hue Nguyen, Iqbal Mohomed, Michael Brudno, Alex Levinshtein, Konstantinos G. Derpanis, Babak Taati, Radek Grzeszczuk

llms reinforcement-learning efficiency multimodal benchmarks

arXiv AI

Aug 12

Hybrid Token Compression for Vision-Language Models

arXiv:2512. 08240v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs.

By Jusheng Zhang, Xiaoyang Guo, Tongyu Mo, Qinhan Lv, Wenhao Chai, Jian Wang, Keze Wang, Liang Lin

llms efficiency multimodal benchmarks

arXiv AI

Jul 28

GOTS: Greedy Orthogonal Token Selection for High-Resolution Vision-Language Models

arXiv:2607. 23913v1 Announce Type: new Abstract: Modern vision-language models (VLMs) increasingly rely on dynamic or high-resolution visual encoding, producing thousands of visual tokens that substantially increase downstream language-model inference cost.

By Jun Ling, Tao Huang, Junzhuo Liu, Bowen Tang, Peng Wang

llms multimodal benchmarks

arXiv AI

Jun 2

AdaCodec: A Predictive Visual Code for Video MLLMs

arXiv:2606. 02569v1 Announce Type: cross Abstract: Video is temporally redundant: adjacent frames usually share most objects, background, and layout.

By Haowen Hou, Zhen Huang, Zheming Liang, Qingyi Si, Chenglin Li, Shuai Dong, Kele Shao, Ruilin Li, Dianyi Wang, Nan Duan, Jiaqi Wang

llms multimodal benchmarks

Related stories

On the Limits of Token Reduction for Efficient Unified Vision Language Training

StreamingVLM: Real-Time Understanding for Infinite Video Streams

AVIS: Adaptive Test-Time Scaling for Vision-Language Models

Hybrid Token Compression for Vision-Language Models

GOTS: Greedy Orthogonal Token Selection for High-Resolution Vision-Language Models

AdaCodec: A Predictive Visual Code for Video MLLMs