arXiv Computation and Language

ExpArt-KG: Artwork Image Description Generation through Iterative Exploration of Knowledge Graphs

The paper introduces ExpArt-KG, a knowledge graph tailored to the artwork domain, and a retrieval‑augmented generation framework that alternates between generating answers and retrieving relevant facts from the graph. By using a correctness judgment to guide the search, the method efficiently gathers the necessary factual information, improving the detail of image explanations while reducing external knowledge retrieval costs. Experimental results demonstrate that the approach maintains generation quality comparable to fixed‑iteration methods.

arXiv Computer Vision

Aug 25

Boosting Knowledge-based Visual Question Answering with Structured Context Reasoning

arXiv:2608.21431v1 Announce Type: new Abstract: Knowledge-based Visual Question Answering aims to answer questions about an image by integrating external knowledge with visual and textual information...

By Qiyou Liu, Yong Zhang, Jianjie Luo, Zhenguo Yang, Yi Yu

llms nlp efficiency multimodal benchmarks

arXiv Computer Vision

Sep 1

Learning to Search: A Decision-Based Agent for Knowledge-Based Visual Question Answering

arXiv:2604.07146v3 Announce Type: replace Abstract: Knowledge-based visual question answering (KB-VQA) requires vision-language models to understand images and use external knowledge, especially for...

By Zhuohong Chen, Zhenxian Wu, Yunyao Yu, Hangrui Xu, Zirui Liao, Zhifang Liu, Xiangwen Deng, Pen Jiao, Haoqian Wang

llms rag agents nlp fine-tuning multimodal benchmarks

arXiv AI

Aug 25

SAFE-G: Structure-aware Faithful Evidence-guided Generation for Knowledge-based Visual Question Answering

arXiv:2608.21796v1 Announce Type: cross Abstract: Knowledge-based Visual Question Answering (KB-VQA) aims to answer queries that necessitate reasoning over external knowledge sources beyond the visua...

By Long Shu, Shuochen Liu, Wei Chen, Junda Lin, Zhi Zheng, Huijun Hou, Tong Xu

llms nlp reinforcement-learning multimodal benchmarks safety

arXiv AI

Sep 10

CrossModalQA: A Cross-modal and Multi-hop Benchmark for Multimodal Retrieval-augmented Generation

arXiv:2609.05518v1 Announce Type: cross Abstract: Despite the strong capabilities of multimodal large language models (MLLMs), their parametric knowledge remains incomplete and difficult to update, m...

By Jiacheng Cai, Zijin Hong, Zheng Yuan, Huachi Zhou, Qinggang Zhang, Xiao Huang

llms rag multimodal benchmarks

arXiv Computation and Language

Sep 30

Question-Specific Knowledge Graphs for Efficient Visual Reasoning

arXiv:2609.35942v1 Announce Type: new Abstract: Recent work in visual question answering has shown that vision-language models can exhibit strong reasoning capabilities by translating visual inputs i...

By Ting-Chih Chen, Emile van Krieken, Shujian Yu, Filip Ilievski

llms nlp reinforcement-learning multimodal benchmarks

arXiv AI

Jul 21

MMGraphRAG: Bridging Vision and Language with Interpretable Multimodal Knowledge Graphs

arXiv:2507. 20804v3 Announce Type: replace Abstract: Large Language Models (LLMs) suffer from hallucinations due to their static parametric knowledge.

By Xueyao Wan, Hang Yu

llms rag multimodal benchmarks safety

arXiv AI

Jul 28

Reason Before You Retrieve: Agentic Planning for Multi-modal RAG

arXiv:2607. 22643v1 Announce Type: new Abstract: Multimodal retrieval-augmented generation (mRAG) aims to answer image-text queries with external knowledge, but most existing systems still retrieve directly from raw multimodal input over a flat evidence space.

By Tianyu Yang, Shir Simon, Zhenzhen Li, Minhao Cheng, Xiangliang Zhang

rag agents fine-tuning multimodal

arXiv Computer Vision

4d ago

ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval

arXiv:2507.21917v2 Announce Type: replace Abstract: Analyzing digitized artworks requires both visual interpretation and deep artistic, contextual, and historical knowledge. We introduce ArtSeek, a m...

By Nicola Fanelli, Gennaro Vessio, Giovanna Castellano

llms rag agents nlp multimodal benchmarks

arXiv AI

Sep 1

Conducting Stylistic Analysis of Paintings through an Art-History Agent

arXiv:2608.29644v1 Announce Type: cross Abstract: Attributing an artwork to an artist has traditionally relied on detailed visual observations and descriptions, known as stylistic analysis in art his...

By Marc S. Walton, Astrid Harth

llms rag agents computer-vision

arXiv AI

Jul 1

Thinking Before Retrieving: Robust Zero-Shot Composed Image Retrieval via Strategic Planning and Self-Criticism

arXiv:2606. 31222v1 Announce Type: new Abstract: Composed image retrieval requires identifying a target image from a gallery by integrating a reference image with a textual modification instruction.

By Gunho Jung, Jeong-Woo Park, Seon Bin Kim, Seong-Whan Lee

rag

arXiv AI

Aug 7

ViSR-KGC: Visual Subgraph Reasoning with Vision-Language Models for Multimodal Knowledge Graph Completion

arXiv:2608. 05833v1 Announce Type: new Abstract: Knowledge graph completion (KGC) aims to infer missing entities or relations from incomplete graph structures, and has evolved into multimodal knowledge graph completion (MMKGC), where entities are associated with multiple modalities such as text and images.

By Jiafan Li, Mengxue Yang, Jiaqi Zhu, Liang Chang, Ying Li, Hongan Wang

llms rag multimodal

arXiv AI

Sep 1

ReGraP-LLaVA: Reasoning enabled Graph-based Personalized Large Language and Vision Assistant

arXiv:2505.03654v3 Announce Type: replace-cross Abstract: Multimodal Large Language Models have shown strong performance across multimodal tasks, and recent personalized MLLMs can recognize user-spec...

By Yifan Xiang, Zhenxi Zhang, Bin Li, Yixuan Weng, Bo Gao, Shoujun Zhou, Yangfan He, Yilin Yuan, Keqin Li

llms multimodal benchmarks

Related stories