arXiv AI

jina-vlm: Small Multilingual Vision Language Model

arXiv:2512. 04032v4 Announce Type: replace-cross Abstract: We present jina-vlm, a token-efficient 2.

arXiv Machine Learning

Jun 30

DataComp-VLM: Improved Open Datasets for Vision-Language Models

arXiv:2606. 28551v1 Announce Type: cross Abstract: Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies.

By Matteo Farina, Vishaal Udandarao, Thao Nguyen, Selim Kuzucu, Maximilian B\"other, Andreas Hochlehnert, Adhiraj Ghosh, Marianna Nezhurina, Karsten Roth, Joschka Struber, Yuhui Zhang, Sebastian Dziadzio, Elaine Sui, Soumya Jahagirdar, Dhruba Ghosh, Hasan Hammoud, Thomas De Min, Simone Caldarella, Jehanzeb Mirza, Sedrick Keh, Mehdi Cherti, Hilde Kuehne, Bernt Schiele, Serena Yeung-Levy, Muhammad Ferjad Naeem, Federico Tombari, Ana Klimovic, Elisa Ricci, Matthias Bethge, Sewoong Oh, Ameya Prabhu, Alessio Tonioni, Jenia Jitsev, Massimiliano Mancini, Ludwig Schmidt, Nikhil Parthasarathy

llms multimodal benchmarks

arXiv AI

Jun 19

The Hidden Evolution of Disguised Visual Context inside the VLM

arXiv:2606. 20077v1 Announce Type: cross Abstract: Visual tokens enter Large Language Models (LLMs) as raw, foreign signals.

By Wish Suharitdamrong, Tony Alex, Muhammad Awais, Sara Atito

llms multimodal benchmarks

arXiv AI

Jun 9

Multilingual Training and Evaluation Resources for Vision-Language Models

arXiv:2604. 18347v2 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) achieved rapid progress in the recent years.

By Daniela Baiamonte, Elena Fano, Matteo Gabburo, Stefano Simonazzi, Leonardo Rigutini, Andrea Zugarini

llms multimodal benchmarks

Hugging Face Trending Papers

Jul 27

MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning

Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency.

llms computer-vision efficiency multimodal benchmarks

arXiv Machine Learning

Jun 4

Stateful Visual Encoders for Vision-Language Models

arXiv:2606. 04433v1 Announce Type: cross Abstract: Vision-language models (VLMs) are increasingly used in multi-image, multi-turn agentic settings where decisions depend on visual changes.

By Zirui Wang, Junwei Yu, Adam Yala, David M. Chan, Joseph E. Gonzalez, Trevor Darrell

llms agents fine-tuning multimodal

arXiv AI

Jul 31

Decoupled Visual Processing: Efficient Multimodal Adaptation via Modality-Specific Transformer Substitution

arXiv:2607. 26596v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities by integrating visual and textual understanding within a unified transformer architecture.

By Mingkuan Feng, Zhengqi Wen, Jianhua Tao

llms fine-tuning multimodal benchmarks

arXiv AI

Jun 2

On the Limits of Token Reduction for Efficient Unified Vision Language Training

arXiv:2606. 01503v1 Announce Type: cross Abstract: Unified vision-language models (VLMs) integrate visual understanding and visual generation within a single autoregressive backbone, but their joint training is computationally expensive and largely overlooked from an efficiency perspective.

By Siyi Chen, Weiming Zhuang, Jingtao Li, Lingjuan Lv

llms multimodal

arXiv AI

Aug 12

Hybrid Token Compression for Vision-Language Models

arXiv:2512. 08240v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) rely on hundreds of visual tokens, leading to high computational and memory costs.

By Jusheng Zhang, Xiaoyang Guo, Tongyu Mo, Qinhan Lv, Wenhao Chai, Jian Wang, Keze Wang, Liang Lin

llms efficiency multimodal benchmarks

arXiv AI

Jul 2

Stateful Token Reduction for Long-Video Hybrid VLMs

arXiv:2603. 00198v2 Announce Type: replace-cross Abstract: Token reduction accelerates long-video vision--language models (VLMs), but existing methods target Transformers, where reduction is treated as token pruning.

By Jindong Jiang, Amala Sanjay Deshmukh, Kateryna Chumachenko, Karan Sapra, Zhiding Yu, Guilin Liu, Andrew Tao, Pavlo Molchanov, Jan Kautz, Wonmin Byeon

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 24

Listening makes Vision Clear for VLMs

arXiv:2606. 23763v1 Announce Type: cross Abstract: Recent work typically assesses vision--language consistency using attention distributions of answer-side tokens.

By Yiyang Chen, Yixin Tan, Binrui Shen

safety

arXiv Machine Learning

Aug 10

Prune Once: Retraining-Free Task-Agnostic Pruning for Vision-Language Models

arXiv:2608. 06901v1 Announce Type: cross Abstract: Vision-language models (VLMs) have achieved remarkable generalization across diverse multimodal tasks through large-scale pre-training, yet their rapidly increasing computational and memory requirements pose significant challenges for deployment in constrained environments.

By Minseok Kang, Hyunwoo Kim, Chanyoung Kim, Minwoo Kim, Jaekoo Lee, Dahuin Jung

llms efficiency multimodal

arXiv AI

5d ago

Vision-Language Models are Fragile Multilingual Associators

arXiv:2608. 12333v1 Announce Type: cross Abstract: Vision-language models must associate visual entities with textual attributes.

By Ritabrata Chakraborty, Rajatsubhra Chakraborty, Shivakumara Palaiahnakote, Angelo Cangelosi, Umapada Pal

llms multimodal benchmarks

Related stories