arXiv Computation and Language October 5, 2026 By Hang Gao, Wujiang Xu, Zhixing Zhang, Kai Mei, Jingyi Yang, Dimitris N. Metaxas

CLIMB: Confidence-Guided Complementary Evidence for Multimodal Retrieval-Augmented Generation

Read the original on arXiv Computation and Language →

CLIMB is a training‑free inference‑time framework for multimodal retrieval‑augmented generation. It builds a compact complementary evidence pool using an MMR‑style objective that balances relevance and redundancy, then refines answers with a confidence‑controlled critic that scores relevance, specificity, and cross‑modal alignment. The method stops refinement when confidence no longer rises, improving performance on Encyclopedic‑VQA and InfoSeek without altering the retriever or language model.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv AI

Sep 10

CrossModalQA: A Cross-modal and Multi-hop Benchmark for Multimodal Retrieval-augmented Generation

arXiv:2609.05518v1 Announce Type: cross Abstract: Despite the strong capabilities of multimodal large language models (MLLMs), their parametric knowledge remains incomplete and difficult to update, m...

By Jiacheng Cai, Zijin Hong, Zheng Yuan, Huachi Zhou, Qinggang Zhang, Xiao Huang

llms rag multimodal benchmarks

arXiv AI

Sep 16

Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models

arXiv:2609.16795v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) can answer knowledge-intensive visual questions by combining visual evidence from images with facts retrieved...

By Zhenbin Wang, Lei Zhang, Lituan Wang, Wei Huang, Yan Wang, Zhenwei Zhang

llms multimodal benchmarks

arXiv AI

Aug 7

Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

arXiv:2604. 01280v2 Announce Type: replace-cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) requires Multimodal Large Language Models (MLLMs) to identify and combine fine-grained visual cues with retrieved textual evidence.

By Marco Morini, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

llms nlp multimodal benchmarks

arXiv AI

Jul 28

Reason Before You Retrieve: Agentic Planning for Multi-modal RAG

arXiv:2607. 22643v1 Announce Type: new Abstract: Multimodal retrieval-augmented generation (mRAG) aims to answer image-text queries with external knowledge, but most existing systems still retrieve directly from raw multimodal input over a flat evidence space.

By Tianyu Yang, Shir Simon, Zhenzhen Li, Minhao Cheng, Xiangliang Zhang

rag agents fine-tuning multimodal

arXiv AI

Jun 26

MKG-RAG-Bench: Benchmarking Retrieval in Multimodal Knowledge Graph-Augmented Generation

arXiv:2606. 26458v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) over knowledge graphs has emerged as a promising approach for grounding large language models, yet existing benchmarks largely overlook the challenges of retrieval in multimodal knowledge graph RAG (MKG-RAG).

By Xiaochen Wang, Bao Hoang, Han Liu, Ting Wang, Fenglong Ma

llms rag multimodal benchmarks

arXiv AI

Aug 18

GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG

arXiv:2608. 15056v1 Announce Type: new Abstract: Multimodal retrieval-augmented generation (RAG) systems often rely on long unstructured contexts or aggressively expanded evidence graphs, which can introduce noisy evidence, weaken multi-hop reasoning, and increase unsupported generation.

By Zafar Ali, Asad Khan, Aalia Malik, Pavlos Kefalas

llms rag multimodal

One email a morning, machine-written

Related stories

CrossModalQA: A Cross-modal and Multi-hop Benchmark for Multimodal Retrieval-augmented Generation

Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models

Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

Reason Before You Retrieve: Agentic Planning for Multi-modal RAG

MKG-RAG-Bench: Benchmarking Retrieval in Multimodal Knowledge Graph-Augmented Generation

GraphLoom: Reliability-Calibrated Graph Evidence Routing for Multimodal KG-RAG