The paper investigates why visual instance retrieval often fails when the same object appears at different apparent sizes in query and gallery images. It identifies object-to-image (O2I) ratio mismatch—where the object occupies different fractions of the two images—as the dominant cause of performance degradation, rather than resolution loss. Experiments on a controlled benchmark with 3,021 Objaverse objects show that over 80% of cross-distance degradation is due to O2I mismatch for most pretrained backbones, and that multi-scale architectures only reduce resolution effects but remain vulnerable to O2I mismatch. The authors demonstrate that query-side scale augmentation and an OWLv2 crop reranker achieve state‑of‑the‑art results on ILIAS 100M without retraining the gallery, and that a LoRA fine‑tune can match these gains in a single forward pass, indicating that O2I robustness can be learned.
By Boaz Meivar, Ofir Kedem, Amit Edenzon, Gal Chechik, Shai Avidan
The paper introduces CERES, a closed‑loop multimodal indexing framework that addresses semantic collapse in multimodal generation by building a three‑level semantic pyramid and using scale‑routed cross‑attention to generate images that remain retrievable by their original queries. CERES employs a co‑occurrence‑aware router, a lightweight U‑Net generator, and a soft‑Jaccard coverage objective to ensure generated images cover the intended concepts, verified by re‑indexing with a frozen vision‑language model and an external DINOv2 probe. Experiments on four pansharpening benchmarks show state‑of‑the‑art performance, especially under extreme scale variation, and significant improvements in concept‑query retrieval and image‑text ranking metrics.
By Guangyuan Dong, Chuang Liu, Haoyu Wang, Yangchen Zeng, Jiaqi Zhang, Li Jiuxing, Xiaoyang Yu, Pinlong Zhao, Yuchao Hou, Ziwei Li, Zheng Lin, Alexander Lim Han Yang, Yusen Wu
Cache-based test-time adaptation improves CLIP predictions by storing and retrieving examples from the target stream while keeping the model frozen. However, existing methods largely treat the feature...
arXiv:2608. 20810v1 Announce Type: cross Abstract: Multimodal information systems increasingly route generated visual content back through the same vision-language index that informed its production, so the output must remain retrievable by the queries it was meant to serve.
By Guangyuan Dong, Chuang Liu, Yangchen Zeng, Haoyu Wang, Xiaoyang Yu, Pinlong Zhao, Yuchao Hou, Ziwei Li, Zheng Lin
FAVE (Foveated Adaptive Visual Encoding) is a lightweight, variable‑resolution Vision Transformer that encodes user‑selected image regions at high acuity while maintaining the image’s native geometry. In controlled experiments on small‑object ImageNet crops, FAVE outperforms a fixed‑resolution ViT by 9.4 top‑1 points while using 12.7× fewer FLOPs. When added as a local branch to FastVLM, FAVE improves TextVQA by 1.60 points and GQA attribute accuracy by 1.31 points, achieving a 3.3× speedup over SmolVLM2-2.2B with only 16 extra local tokens.
By Amitangshu Mukherjee, Kaushik Roy
The paper investigates how the choice of retrieval encoder affects cache‑based test‑time adaptation for CLIP. By keeping the memory fixed and varying the retrieval space across sixteen encoders, the authors show that retrieval space can dramatically alter performance, with gains ranging from +0.44 to +19.7 points on ImageNet‑A. They introduce MARC, a training‑free system that pairs frozen CLIP with DINOv2‑B for retrieval, achieving superior out‑of‑distribution accuracy and efficiency compared to prior methods.
By Mahir Shahriar Tamim, Md. Samiul Alim, Azmine Toushik Wasi, Shahriyar Zaman Ridoy, Meharun Nesa, Mohammad Abu Yousuf, Alex Lamb, Mohammad Ali Moni
MINER is a training‑free inference framework that enhances frozen dual‑encoder models for text‑to‑image retrieval when queries refer to small, visually subordinate objects in cluttered scenes. It augments the global image embedding with a bank of region‑level embeddings and applies hubness‑correcting similarity rescoring, thereby recovering visual evidence that global pooling underweights. The authors introduce ROCS, a benchmark derived from Flickr30K and MS COCO, and demonstrate that MINER improves retrieval performance across CLIP, SigLIP, and SigLIP 2 backbones on both ROCS and standard splits, attributing gains mainly to broader spatial coverage rather than precise crop placement.
By Abdulmalik Alquwayfili, Faisal AlMeshal, Jumanah Almajnouni, Huda Abdulhadi Alamri, Muhammad Kamran J Khan
arXiv:2610.09313v1 Announce Type: new
Abstract: Vision-language models (VLMs) often miss small objects in large images. We ask three questions: what limits them, which of these limits better models c...
By Junzhe Shi, Yuan Gan, Shida Jiang
arXiv:2410.21582v4 Announce Type: replace-cross
Abstract: Large-scale pretrained models are widely leveraged as foundations for learning new specialized tasks via fine-tuning, with the goal of mainta...
By Jaedong Hwang, Brian Cheung, Zhang-Wei Hong, Akhilan Boopathy, Pulkit Agrawal, Ila Fiete
The paper reviews 50 image augmentation and generation techniques, categorizing them into ten groups, and conducts a large‑scale empirical study to assess their effectiveness as test generators for embedding‑based image retrieval systems. Using Amazon Titan and OpenCLIP embeddings, the authors evaluate the techniques across four dimensions—embedding‑space similarity, embedding uncertainty, semantic realism, and retrieval failure rate—on CIFAR‑10, ImageNet‑1K, and an industrial dataset. Results show that weather simulation and SaSPA yield the highest uncertainty and failure rates while maintaining realistic visuals, whereas GAN‑based methods produce low realism due to synthetic artifacts.
By Yehan De Silva, Anirudh Sridhar, Armin Lotfy, Nafiseh Kahani, Yvan Labiche, Ziyu Wang, Frank Ouyang, Clare Carty, Azalia Shamsaei
arXiv:2608.29917v1 Announce Type: new
Abstract: Personalized segmentation and personalized retrieval both aim to identify the same physical object across different images. While the former localizes...
By Gabriele Trivigno, Marcos Alfaro, Claudia Cuttano, Gabriele Berton, Luis Pay\'a, Carlo Masone
arXiv:2606. 26734v1 Announce Type: cross Abstract: The impact of real-world noise on Open Vocabulary Object Detectors (OV-ODs) remains poorly understood due to their architectural complexity.
By Priyank Pathak, Mukilan Karuppasamy, Aaditya Baranwal, Shruti Vyas, Yogesh S Rawat