TwinICL is a procedurally generated benchmark that pairs matched text and image versions of tasks to enable controlled comparison of in‑context learning (ICL) across modalities. Experiments on six open‑weight models and 38 tasks show that multimodal ICL consistently underperforms text‑only ICL, with varying gaps by task family. Interventions targeting visual access, task framing, and reasoning can recover strong multimodal performance on a diagnostic subset, yet a modality gap remains even when explicit task instructions are provided, highlighting the dual role of demonstrations as context and evidence.
LLMs have shown strong in-context learning (ICL) capability, but extending it to Multimodal LLMs (MLLMs) remains challenging. Prior multimodal ICL methods often rely on ICL-specific datasets for addit...
The paper introduces ProCoRe, a contrastive reinforcement learning approach that improves visual grounding in multimodal large language models (MLLMs) for in-context learning (ICL). By generating captions for contrastive image pairs, decomposing them into propositions, and optimizing contrastive rewards, ProCoRe enhances alignment with relevant visual evidence without requiring ICL-formatted training data. The method achieves significant gains in multimodal ICL classification accuracy and captioning ROUGE-L over the Qwen-VL-3-8B baseline and outperforms state‑of‑the‑art ICL‑data‑dependent methods across multiple benchmarks. Additionally, the authors present PICL, a personalized multimodal ICL captioning benchmark designed to test genuine understanding of visual demonstrations.
The paper introduces AVA, a training‑free framework that improves interleaved multimodal in‑context learning for frozen large vision‑language models. AVA decomposes the interleaved context into per‑sample threads, selects influential demonstrations while removing redundancy, and calibrates consistency using a weak sibling model’s confidence. Across five vision‑language benchmarks, AVA raises performance by up to 2.4 points, narrows the visual performance gap on ScienceQA‑IMG, and remains robust even with 30% noisy demonstrations, all while keeping the LVLM frozen and adding only modest latency.
arXiv:2609.15683v1 Announce Type: new Abstract: While In-Context Learning (ICL) enables models to adapt from exemplars without parameter updates, multimodal ICL remains largely underexplored, particu...
arXiv:2606. 04434v1 Announce Type: cross Abstract: Multimodal In-Context Learning (ICL) has emerged as a practical inference paradigm for Multimodal Large Language Models, where a small set of interleaved image-text In-Context Demonstrations (ICDs) conditions the model to solve new tasks.
arXiv:2511. 16107v3 Announce Type: replace-cross Abstract: Visual in-context learning (VICL) solves visual tasks by conditioning on a few input-output demonstrations without any model training.
The paper introduces a multimodal in‑context learning framework that uses contrastive demonstration modeling to align large language models’ responses with the required reasoning paths. By contrasting suboptimal and better responses and incorporating a response‑conditioned retrieval mechanism, the method explicitly guides models beyond surface imitation. Experiments on various multimodal tasks, especially visual question answering, show consistent performance gains.
arXiv:2602.02326v3 Announce Type: replace Abstract: If multilingual language model representations contain separable language-associated structure, then shifting between languages may require only si...
arXiv:2610.01054v1 Announce Type: cross Abstract: In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requi...
The paper investigates when in‑context learning (ICL) can succeed even if the instruction or target task is not covered by a model’s pretraining data. It introduces a theoretical framework that distinguishes two types of distribution shift—unsupported instructions and unsupported target tasks—and shows how demonstrations can mitigate these shifts. The authors further extend the analysis to long‑horizon tasks that can be decomposed into pretraining‑supported subtasks, providing error bounds that explain how ICL can handle tasks beyond the model’s original support.
arXiv:2609.39920v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) exhibit strong multimodal in-context learning (ICL) capabilities, yet this ability degrades substantially as mod...
arXiv:2606. 19965v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) are increasingly expected to act on visual information, yet the same scene may require different actions under different task contexts.