arXiv Computer Vision September 14, 2026 By Bingchen Huang, Zhiling Wang, Yifu Chen, Yuanchao Du

Retrieved Images as Visual Thought: Training-Free Multimodal In-Context Learning for the Open-vs-Closed Gap

Read the original on arXiv Computer Vision →

ReVisIT is a train‑free framework that turns retrieved image‑label pairs into units of visual thought, combining structured class definitions, multimodal retrieval, and alternating user/assistant injection before joint decoding. On several benchmarks—including Fast Open MiniImageNet, Bongard‑OpenWorld, and the newly released MAAC‑Bench—ReVisIT achieves performance comparable to or surpassing large, trained models while using far fewer parameters. The approach demonstrates that high‑quality retrieval and a simple turns layer can provide a universal performance boost across diverse multimodal tasks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv AI

Jul 8

Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAG

arXiv:2607. 05438v1 Announce Type: cross Abstract: Multimodal retrieval-augmented generation (RAG) grounds a generator in evidence drawn from heterogeneous modalities -- text, tables, and images.

By Xue Li, Yiming Gai

llms rag multimodal

arXiv Computer Vision

4d ago

How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

arXiv:2605.27310v2 Announce Type: replace Abstract: Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they reason in language and discard the fine-grained geometry t...

By Qian Yang, Ankur Sikarwar, Huy Le, Le Zhang, Zhuan Shi, Perouz Taslakian, Aishwarya Agrawal

llms multimodal benchmarks

arXiv AI

Jun 16

Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings

arXiv:2606. 15134v1 Announce Type: cross Abstract: Vision encoders for retrieval are typically trained with class-label supervision: each training pair reduces to a scalar that uniformly pushes the embedding apart or pulls it together, as if every visual attribute either differed or matched.

By Shubhang Bhatnagar, Dheeraj Baiju, Narendra Ahuja

llms rag efficiency multimodal benchmarks

arXiv AI

Aug 19

Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models

The paper investigates how new concepts can be integrated into unified multimodal models (UMMs) by separating generation and understanding objectives through a novel visual entity bound to a single task direction. Experiments show that the effectiveness of cross‑task usability depends on where the concept is injected into the shared computation, with a mid‑stack alignment objective achieving high concept acquisition with minimal loss to overall performance. The study highlights that unified weights alone are insufficient; the two directions must share a semantic format at the entry point for efficient concept integration.

By Zongyang Qiu, Yihan Wu, Kaixuan Fan, Bo Li, Hui Xiong

diffusion multimodal safety

arXiv Computer Vision

Sep 1

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs

arXiv:2608.30705v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) struggle with fine-grained Visual Search, the task of locating small or rare objects in high-resolution images...

By Jingyi He, Sanghwan Kim, Zeynep Akata

llms reinforcement-learning multimodal

arXiv Computer Vision

Sep 1

Instruction Distillation: Text Instructions as Visual Examples

arXiv:2608.28696v1 Announce Type: new Abstract: Visual in-context learning (ICL) with multimodal large language models (MLLMs) is effective for fine-grained visual classification, but each retrieved...

By Hardik Jindal, Soumyabrata Pal, Sayak Ray Chowdhury

llms efficiency multimodal benchmarks

One email a morning, machine-written

Related stories

Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAG

How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning

Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings

Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs

Instruction Distillation: Text Instructions as Visual Examples