arXiv AI
Jun 24
Listening makes Vision Clear for VLMs
arXiv:2606. 23763v1 Announce Type: cross Abstract: Recent work typically assesses vision--language consistency using attention distributions of answer-side tokens.
Test-Time Training for Modality Order Consistency in Vision-Language Models
We find that vision-language models are sensitive to a specific semantically irrelevant change: the order in which the image and question are presented. Across three models and three benchmarks, image first prompting consistently outperforms question-first prompting, revealing a repeatable modality order failure.
Related stories
arXiv Machine Learning
Jul 7
USE: A Unified Self-Ensembling Framework for Test-Time Prompt Tuning
arXiv:2607. 03900v1 Announce Type: cross Abstract: Test-time adaptation (TTA) has emerged as a popular paradigm for improving the performance of vision-language models (e.
arXiv AI
Jun 19
The Hidden Evolution of Disguised Visual Context inside the VLM
arXiv:2606. 20077v1 Announce Type: cross Abstract: Visual tokens enter Large Language Models (LLMs) as raw, foreign signals.
arXiv Machine Learning
Jul 2
StochasT: Learning with Stochastic Turn Depth for Visual Instruction Tuning
arXiv:2607. 00465v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) rely extensively on Visual Instruction Tuning (VIT) to elicit their multimodal reasoning capabilities.
arXiv AI
Jul 28
Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimization
arXiv:2607. 24354v1 Announce Type: new Abstract: Automatic prompt optimization (APO) has been widely adopted to adapt vision-language models (VLMs) to downstream tasks without weight updates, yielding promising results.
arXiv Machine Learning
Jun 5
Test-Time Training for Visual Foresight Vision-Language-Action Models
arXiv:2605. 08215v2 Announce Type: replace-cross Abstract: Visual Foresight VLA (VF-VLA) has become a prominent architectural choice in the recent VLA due to its impressive performance.
arXiv AI
Jun 8
SS-TPT: Stability and Suitability-Guided Test-Time Prompt Tuning for Adversarially Robust Vision-Language Models
arXiv:2606. 06943v1 Announce Type: cross Abstract: Vision-language models (VLMs) such as CLIP achieve strong zero-shot recognition but remain highly fragile under adversarial perturbations.
arXiv Machine Learning
Jul 2
MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation
arXiv:2602. 21397v2 Announce Type: replace-cross Abstract: Prompt learning has become a dominant paradigm for adapting vision-language models (VLMs) such as CLIP to downstream tasks without modifying pretrained weights.
arXiv AI
Jun 4
EvoPrompt: Guided Prompt Evolution for Vision-Language Models Adaptation
arXiv:2603. 09493v2 Announce Type: replace-cross Abstract: The adaptation of large-scale vision-language models (VLMs) to downstream tasks with limited labeled data remains a significant challenge.
arXiv Machine Learning
Jul 13
Robustifying Vision-Language Models via Test-Time Prompt Adaptation
arXiv:2607. 09450v1 Announce Type: cross Abstract: Pre-trained Vision-Language Models (VLMs) such as CLIP achieve strong zero-shot generalization, but their performance degrades sharply under adversarial perturbations.
arXiv AI
Jun 11
AVIS: Adaptive Test-Time Scaling for Vision-Language Models
arXiv:2606. 11576v1 Announce Type: cross Abstract: Modern Vision-Language Models (VLMs) benefit from chain-of-thought prompting and test-time scaling, but these gains often come with prohibitive inference cost due to large visual contexts and long decoding chains.
arXiv Machine Learning
Jun 25
Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models
arXiv:2606. 26079v1 Announce Type: cross Abstract: Standard benchmarks for multimodal large language models (MLLMs) score each item on one canonical ordering and miss whether order-irrelevant shuffling changes the answer, a baseline reliability property called for by emerging AI evaluation guidelines.