The paper introduces RSMEB, a unified benchmark for remote‑sensing multimodal retrieval that evaluates both cross‑modal and interleaved retrieval across 21 tasks under a single ranking protocol. It also presents VLM2GeoVec, an instruction‑conditioned single‑encoder model that embeds image, text, bounding‑box, and geo‑coordinate tokens into one sequence and achieves state‑of‑the‑art performance on region‑caption, referring‑expression, and semantic geo‑aware retrieval while remaining competitive on conventional tasks. The authors provide code, checkpoints, and data on GitHub to facilitate reproducibility.
By Emanuel S\'anchez Aimar, Gulnaz Zhambulova, Fahad Shahbaz Khan, Yonghao Xu, Michael Felsberg
arXiv:2610.33551v1 Announce Type: new
Abstract: High-resolution remote sensing (RS) imagery provides fine spatial detail over large areas, making it essential for fine-grained scene understanding. Re...
By Yuxi Li, Lu Si, Xia Liao, Yidong Shen, Zheng Wei, Jun Zhang, Xin Yuan
arXiv:2608. 12987v1 Announce Type: cross Abstract: Generative information retrieval (GIR) has emerged as a compelling alternative to the conventional index-retrieve-then-rank retrieval pipeline by training a generator to produce the identifiers of relevant items directly.
By Kaipeng Li, Haitao Yu, Xuanchen Zhou
arXiv:2609.05518v1 Announce Type: cross
Abstract: Despite the strong capabilities of multimodal large language models (MLLMs), their parametric knowledge remains incomplete and difficult to update, m...
By Jiacheng Cai, Zijin Hong, Zheng Yuan, Huachi Zhou, Qinggang Zhang, Xiao Huang
The paper introduces HFCMA, a hierarchical fine-grained cross‑modal alignment framework that transforms low‑level image patches and text tokens into adaptive visual regions and textual phrases through dynamic granularity discovery. It then performs structured region‑phrase matching using an entropy‑regularized optimal transport formulation, incorporating background debiasing, textual importance weighting, and prototype consistency regularization. Experiments on Flickr30K and MS‑COCO demonstrate that this approach improves fine‑grained image‑text retrieval by providing structured local alignment information and better discrimination of subtle semantic differences.
By Pengfeng Song, Shaozhong Zhang, Pengfei Shao
Due to their strong generalizable multimodal processing and reasoning capabilities, Multimodal Large Language Models (MLLMs) have demonstrated significant potential as universal image retrievers, effectively addressing diverse real-world image retrieval tasks. Nevertheless, pioneering studies, while promising, overlook the potential of fine-grained context modeling and disentangled fine-tuning objectives in enhancing MLLMs' retrieval performance, particularly for complex tasks such as long-text-to-image retrieval, visual dialog retrieval, and composed image retrieval (CIR).
The paper introduces ReT-2, a unified retrieval model that handles multimodal queries containing both images and text and searches across multimodal document collections. It employs a recurrent Transformer architecture with LSTM-inspired gating to integrate information across layers and modalities, capturing fine-grained visual and textual details. Evaluations on M2KR and M-BEIR benchmarks show state‑of‑the‑art performance, faster inference, and lower memory usage, and the model also boosts downstream tasks in retrieval‑augmented generation pipelines.
By Davide Caffagni, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara
The paper evaluates 14 state‑of‑the‑art text‑to‑video retrieval models on three major datasets using a unified framework. It finds that short, clear, and simple captions—especially those describing single actions or color attributes—yield higher recall, while complex, multi‑step events or fine‑grained scene descriptions remain difficult for all models. Attention‑driven architectures better handle temporally dependent or multi‑step queries, whereas dual‑encoder and multimodal fusion models excel mainly on simpler captions, and cross‑dataset generalization improves with larger, more diverse caption sets but generative captions do not consistently boost accuracy.
By Maria-Eirini Pegia, Dimitrios Stefanopoulos, Bj\"orn {\TH}\'or J\'onsson, Anastasia Moumtzidou, Ilias Gialampoukidis, Stefanos Vrochidis, Ioannis Kompatsiaris
Geospatial and urban applications increasingly require models to compare heterogeneous evidence across street-view imagery, remote-sensing observations, text descriptions, region proposals, and temporal change cues. However, existing multimodal embedding models and benchmarks are still largely designed and evaluated around general-purpose image-text matching, leaving unclear whether unified embedding space can support heterogeneous geospatial tasks involving spatial relationships, fine-grained semantics, and temporal changes.
Current instruction-based image retrieval systems are powerful but limited to single-turn interactions, failing to capture the iterative nature of complex, real-world visual searches. To overcome this limitation, we introduce Contextual Composed Image Retrieval (CoCo-IR), a novel task that enables users to progressively refine search results through interactions.
arXiv:2610.32311v1 Announce Type: new
Abstract: Geographic regions are multimodal: satellite imagery captures visual morphology, text conveys human-interpretable semantics, and map structures describ...
By Radoslaw Malek, Szymon Soltysiak, Aleksandra Kawala-Sterniuk, Piotr Szymanski
arXiv:2511. 01390v2 Announce Type: replace-cross Abstract: Fine-grained cross-modal alignment aims to establish precise local correspondences between vision and language, forming a cornerstone for visual question answering and related multimodal applications.
By Xinyu Mao, Junsi Li, Haoji Zhang, Yu Liang, Ming Sun