Traditional image captioning methods often struggle to generate comprehensive, context-rich descriptions, especially for details not directly observable from visual cues. To overcome this, we propose a novel retrieval-augmented image captioning framework that generates captions with deeper insights, such as object attributes, event context, and underlying significance, by leveraging external knowledge.
The paper introduces ReT-2, a unified retrieval model that handles multimodal queries containing both images and text and searches across multimodal document collections. It employs a recurrent Transformer architecture with LSTM-inspired gating to integrate information across layers and modalities, capturing fine-grained visual and textual details. Evaluations on M2KR and M-BEIR benchmarks show state‑of‑the‑art performance, faster inference, and lower memory usage, and the model also boosts downstream tasks in retrieval‑augmented generation pipelines.
By Davide Caffagni, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara
arXiv:2606. 24112v1 Announce Type: new Abstract: Multimodal misinformation detection is increasingly important because viral posts now combine long multilingual narratives, several images, mixed provenance, and subtle text--image framing errors.
By Chenhao Dang, Dantong Zhu, Jun Yang, Conghui He, Weijia Li
arXiv:2606. 28344v1 Announce Type: cross Abstract: Augmenting large language models (LLMs) with retrieved web text has become a dominant paradigm, yet the web is not natively textual: existing systems depend on complex parsing pipelines that linearize HTML and discard layout, visual structure, and formatting.
By Yichuan Wang, Zhifei Li, Zirui Wang, Paul Teiletche, Lesheng Jin, Matei Zaharia, Joseph E. Gonzalez, Sewon Min
The paper introduces a new perspective on entity rarity in multimodal entity linking by using knowledge‑graph structural metrics instead of popularity metrics, revealing many rare entities previously overlooked. Experiments show that state‑of‑the‑art models suffer a 15.4–39.9% accuracy drop on these rare‑entity slices. The authors propose a training‑free framework that combines reasoning and retrieval with a vision‑language model, achieving a 6.9% overall accuracy gain and up to 23.3% improvement on rare entities, and release a new benchmark MERLIN‑Rare for focused evaluation.
By Parinthapat Pengpun, Simran Khanuja, Graham Neubig
arXiv:2512.20257v2 Announce Type: replace
Abstract: With the rise of easily accessible generative tools for creating and manipulating multimedia content, the threat of realistic synthetic alterations...
By Daniele Cardullo, Simone Teglia, Irene Amerini
arXiv:2608.23102v1 Announce Type: new
Abstract: Composed Image Retrieval (CIR) is an emerging paradigm in content-based image retrieval that enables users to formulate compositional queries by combin...
By Fan Xu, Luis A. Leiva
arXiv:2608.22214v1 Announce Type: new
Abstract: In domain-specific multimodal long documents, images and text jointly convey complex knowledge that cannot be fully captured by plain text alone. Howev...
By Yikai Gao, Ding Xia, Xi Yang
arXiv:2609.12678v1 Announce Type: new
Abstract: Short-video platforms have become a primary news source for the public, which has also enabled the widespread dissemination of fake news videos. We stu...
By Yifeng Luo, Yupeng Li, Ming Tang, Jianxiong Guo, Liang Lan
arXiv:2609.05518v1 Announce Type: cross
Abstract: Despite the strong capabilities of multimodal large language models (MLLMs), their parametric knowledge remains incomplete and difficult to update, m...
By Jiacheng Cai, Zijin Hong, Zheng Yuan, Huachi Zhou, Qinggang Zhang, Xiao Huang
Current instruction-based image retrieval systems are powerful but limited to single-turn interactions, failing to capture the iterative nature of complex, real-world visual searches. To overcome this limitation, we introduce Contextual Composed Image Retrieval (CoCo-IR), a novel task that enables users to progressively refine search results through interactions.
arXiv:2510.23508v4 Announce Type: replace
Abstract: Existing real-world datasets for multimodal fact-checking have multiple limitations: they contain few instances, cover only one or two languages, f...
By Jiahui Geng, Jonathan Tonglet, Iryna Gurevych