The paper introduces SAGE, a lightweight method that guides decoder attention in vision-language models away from prompt‑invariant attention sinks (PIS) toward query‑dependent regions of interest (ROIs). By using token‑aligned ROI masks from vision backbones like CLIP, ViT, and DINOv3, SAGE improves visual grounding and reduces hallucinations across various VLM families. Experiments show consistent gains on public vision‑language benchmarks, especially in fine‑grained visual discrimination tasks where localized evidence is essential.
By Jeonghyo Song, YoungJoon Yoo
arXiv:2607. 18615v1 Announce Type: cross Abstract: Machine unlearning for vision-language models (VLMs) remains underexplored.
By Zijie Liu, Jinhao Duan, Gaowen Liu, Sijia Liu, Tianlong Chen
Machine unlearning for vision-language models (VLMs) remains underexplored. Unlike language models, VLMs combine a language backbone with visual components, which makes unlearning more complex.
arXiv:2610.27724v1 Announce Type: new
Abstract: Dense linear probes assess task readability from language model (LM) representations, but do not directly reveal how many features suffice for predicti...
By Kaiyang Wan, Yuxia Wang
The paper shows that controlled visual in‑context learning can help vision‑language models (VLMs) overcome errors caused by training priors. By providing demonstrations, the models focus more on visual evidence, yielding significant accuracy gains on counterfactual examples while leaving prior‑aligned performance largely unchanged. This effect is observed across five VLM families and five vision‑centric tasks, indicating that many VLMs possess latent visual grounding abilities that standard tests may miss.
By Kun Wang, Xindi Wu, Sanghyuk Chun, Olga Russakovsky, Esin Tureci
arXiv:2601. 03309v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models, which integrate pretrained large Vision-Language Models (VLM) into their policy backbone, are gaining significant attention for their promising generalization capabilities.
By Jianke Zhang, Xiaoyu Chen, Qiuyue Wang, Mingsheng Li, Yanjiang Guo, Yucheng Hu, Jiajun Zhang, Shuai Bai, Junyang Lin, Jianyu Chen
The paper introduces the Capability-Driven Multimodal Scaling Law, a cross-family framework that predicts vision-language model (VLM) benchmark accuracy from a low-dimensional textual capability score extracted via PCA. By training over 150 VLMs on 34 large language models across seven families, the authors demonstrate that the law accurately extrapolates transfer rates from 8B to 72B‑parameter backbones, predicts full training trajectories, and generalizes to unseen model families. The study also reveals actionable insights, such as certain textual benchmarks negatively correlating with multimodal performance and base LLMs outperforming instruction-tuned counterparts as VLM backbones due to higher absorption rates.
By Ziran Li, Qiang Wang, Zhengyu Chen, Shanglin Lei, Borun Chen, Jingang Wang, Xunliang Cai
arXiv:2609.18084v1 Announce Type: cross
Abstract: Fine-tuning a Vision-Language-Action (VLA) model for a new deployment environment is expensive, yet most methods apply uniform-capacity adapters to e...
By Shahram Najam Syed, Arthur Jakobsson, Prayuj Sachdev, Jeffrey Ichnowski
The paper revisits the use of contrastive loss for post‑training CLIP models, showing that the reported catastrophic forgetting is mainly due to an overly large temperature parameter in the InfoNCE objective. By setting a smaller temperature, contrastive post‑training can actually improve CLIP. The authors introduce ComCLIP, a lightweight single‑epoch recipe that freezes the text encoder, trains the vision encoder with a properly tempered contrastive loss, an MSE anchoring loss, and a relational distillation loss, achieving comparable zero‑shot classification to CLIP‑Refine while significantly enhancing visual feature transferability and maintaining downstream compatibility with LLaVA.
By Zidan Wang, Yaqian Li, Xiaokai Zhang, Kaiwen Long, Kun He, Hanpeng Liu
arXiv:2610.03389v1 Announce Type: cross
Abstract: Vision language models (VLMs) incur substantial inference cost because every visual token is processed by the attention and MLP projections of every...
By Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen, Shaoyi Huang, Pu Zhao, Yanzhi Wang
arXiv:2610.15855v1 Announce Type: new
Abstract: We study how to detect and mitigate visual attention sinks in vision-language models (VLMs). Visual attention sinks are image tokens that absorb a larg...
By Dengjia Zhang, Alexander Martin, Pengyu Yan, Xihan Sun, Jiaqi Li, Benjamin Van Durme, Andrew Yates, Reno Kriz, William Walden
arXiv:2607. 09438v1 Announce Type: cross Abstract: Test-time scaling (TTS) reliably improves reasoning in large language models, but whether it transfers to small open vision-language models remains unclear.
By Spiros Baxevanakis, Peng-Jian Yang