arXiv AI
Multimodal In-Context Learning with Explicit Rule Induction
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv Machine Learning
Sep 15
TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals
TwinICL is a procedurally generated benchmark that pairs matched text and image versions of tasks to enable controlled comparison of in‑context learning (ICL) across modalities. Experiments on six open‑weight models and 38 tasks show that multimodal ICL consistently underperforms text‑only ICL, with varying gaps by task family. Interventions targeting visual access, task framing, and reasoning can recover strong multimodal performance on a diagnostic subset, yet a modality gap remains even when explicit task instructions are provided, highlighting the dual role of demonstrations as context and evidence.
arXiv Machine Learning
Jun 4
Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learning
arXiv:2606. 04434v1 Announce Type: cross Abstract: Multimodal In-Context Learning (ICL) has emerged as a practical inference paradigm for Multimodal Large Language Models, where a small set of interleaved image-text In-Context Demonstrations (ICDs) conditions the model to solve new tasks.
arXiv Computation and Language
Sep 1
GUIDE: Guiding Internal Evidence with Language Instructions
arXiv:2608.30712v1 Announce Type: new Abstract: Large multimodal models follow instructions about what to generate, but not necessarily about what evidence to rely on. Hence, models may continue to d...
arXiv AI
Sep 11
Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context Learning
The paper introduces a multimodal in‑context learning framework that uses contrastive demonstration modeling to align large language models’ responses with the required reasoning paths. By contrasting suboptimal and better responses and incorporating a response‑conditioned retrieval mechanism, the method explicitly guides models beyond surface imitation. Experiments on various multimodal tasks, especially visual question answering, show consistent performance gains.
arXiv Computer Vision
Oct 23
Visual Grounding First, Multimodal In-context Learning Follows
The paper introduces ProCoRe, a contrastive reinforcement learning approach that improves visual grounding in multimodal large language models (MLLMs) for in-context learning (ICL). By generating captions for contrastive image pairs, decomposing them into propositions, and optimizing contrastive rewards, ProCoRe enhances alignment with relevant visual evidence without requiring ICL-formatted training data. The method achieves significant gains in multimodal ICL classification accuracy and captioning ROUGE-L over the Qwen-VL-3-8B baseline and outperforms state‑of‑the‑art ICL‑data‑dependent methods across multiple benchmarks. Additionally, the authors present PICL, a personalized multimodal ICL captioning benchmark designed to test genuine understanding of visual demonstrations.
Hugging Face Trending Papers
Oct 22
Visual Grounding First, Multimodal In-context Learning Follows
LLMs have shown strong in-context learning (ICL) capability, but extending it to Multimodal LLMs (MLLMs) remains challenging. Prior multimodal ICL methods often rely on ICL-specific datasets for addit...