The paper identifies that visual instance retrieval failures are mainly caused by object-to-image (O2I) ratio mismatch rather than resolution loss, as shown on a benchmark of 3,021 Objaverse objects. Multi-scale architectures reduce resolution effects but still suffer from O2I mismatch, which is asymmetric across query and gallery sizes. The authors propose query-side scale augmentation and an OWLv2 crop reranker, achieving state‑of‑the‑art results on ILIAS 100M, and demonstrate that a LoRA fine‑tune can match these gains in a single forward pass.
The paper introduces CERES, a closed‑loop multimodal indexing framework that addresses semantic collapse in multimodal generation by building a three‑level semantic pyramid and using scale‑routed cross‑attention to generate images that remain retrievable by their original queries. CERES employs a co‑occurrence‑aware router, a lightweight U‑Net generator, and a soft‑Jaccard coverage objective to ensure generated images cover the intended concepts, verified by re‑indexing with a frozen vision‑language model and an external DINOv2 probe. Experiments on four pansharpening benchmarks show state‑of‑the‑art performance, especially under extreme scale variation, and significant improvements in concept‑query retrieval and image‑text ranking metrics.
By Guangyuan Dong, Chuang Liu, Haoyu Wang, Yangchen Zeng, Jiaqi Zhang, Li Jiuxing, Xiaoyang Yu, Pinlong Zhao, Yuchao Hou, Ziwei Li, Zheng Lin, Alexander Lim Han Yang, Yusen Wu
FAVE (Foveated Adaptive Visual Encoding) is a lightweight, variable‑resolution Vision Transformer that encodes user‑selected image regions at high acuity while maintaining the image’s native geometry. In controlled experiments on small‑object ImageNet crops, FAVE outperforms a fixed‑resolution ViT by 9.4 top‑1 points while using 12.7× fewer FLOPs. When added as a local branch to FastVLM, FAVE improves TextVQA by 1.60 points and GQA attribute accuracy by 1.31 points, achieving a 3.3× speedup over SmolVLM2-2.2B with only 16 extra local tokens.
By Amitangshu Mukherjee, Kaushik Roy
Cache-based test-time adaptation improves CLIP predictions by storing and retrieving examples from the target stream while keeping the model frozen. However, existing methods largely treat the feature...
arXiv:2608. 20810v1 Announce Type: cross Abstract: Multimodal information systems increasingly route generated visual content back through the same vision-language index that informed its production, so the output must remain retrievable by the queries it was meant to serve.
By Guangyuan Dong, Chuang Liu, Yangchen Zeng, Haoyu Wang, Xiaoyang Yu, Pinlong Zhao, Yuchao Hou, Ziwei Li, Zheng Lin
The paper investigates how the choice of retrieval encoder affects cache‑based test‑time adaptation for CLIP. By keeping the memory fixed and varying the retrieval space across sixteen encoders, the authors show that retrieval space can dramatically alter performance, with gains ranging from +0.44 to +19.7 points on ImageNet‑A. They introduce MARC, a training‑free system that pairs frozen CLIP with DINOv2‑B for retrieval, achieving superior out‑of‑distribution accuracy and efficiency compared to prior methods.
By Mahir Shahriar Tamim, Md. Samiul Alim, Azmine Toushik Wasi, Shahriyar Zaman Ridoy, Meharun Nesa, Mohammad Abu Yousuf, Alex Lamb, Mohammad Ali Moni
MINER is a training‑free inference framework that enhances frozen dual‑encoder models for text‑to‑image retrieval when queries refer to small, visually subordinate objects in cluttered scenes. It augments the global image embedding with a bank of region‑level embeddings and applies hubness‑correcting similarity rescoring, thereby recovering visual evidence that global pooling underweights. The authors introduce ROCS, a benchmark derived from Flickr30K and MS COCO, and demonstrate that MINER improves retrieval performance across CLIP, SigLIP, and SigLIP 2 backbones on both ROCS and standard splits, attributing gains mainly to broader spatial coverage rather than precise crop placement.
By Abdulmalik Alquwayfili, Faisal AlMeshal, Jumanah Almajnouni, Huda Abdulhadi Alamri, Muhammad Kamran J Khan
arXiv:2610.09313v1 Announce Type: new
Abstract: Vision-language models (VLMs) often miss small objects in large images. We ask three questions: what limits them, which of these limits better models c...
By Junzhe Shi, Yuan Gan, Shida Jiang
The paper reviews 50 image augmentation and generation techniques, categorizing them into ten groups, and conducts a large‑scale empirical study to assess their effectiveness as test generators for embedding‑based image retrieval systems. Using Amazon Titan and OpenCLIP embeddings, the authors evaluate the techniques across four dimensions—embedding‑space similarity, embedding uncertainty, semantic realism, and retrieval failure rate—on CIFAR‑10, ImageNet‑1K, and an industrial dataset. Results show that weather simulation and SaSPA yield the highest uncertainty and failure rates while maintaining realistic visuals, whereas GAN‑based methods produce low realism due to synthetic artifacts.
By Yehan De Silva, Anirudh Sridhar, Armin Lotfy, Nafiseh Kahani, Yvan Labiche, Ziyu Wang, Frank Ouyang, Clare Carty, Azalia Shamsaei
arXiv:2410.21582v4 Announce Type: replace-cross
Abstract: Large-scale pretrained models are widely leveraged as foundations for learning new specialized tasks via fine-tuning, with the goal of mainta...
By Jaedong Hwang, Brian Cheung, Zhang-Wei Hong, Akhilan Boopathy, Pulkit Agrawal, Ila Fiete
The paper introduces TFA, a training‑free aggregation technique that calibrates frozen visual foundation models for visual place recognition. TFA uses cross‑codebook agreement, retrieval coverage, and spectral statistics to adjust residual assignment, spectral shaping, and global‑feature fusion without requiring place labels or task‑specific weights. Experiments with a DINOv2‑B backbone show significant Recall@1 gains over existing training‑free methods across multiple benchmarks, demonstrating that reliability‑guided aggregation can unlock additional retrieval performance from frozen representations.
By Xin Li, Zhimin Mao, Shang Wang, Siyuan Duan, Geng Zhang
arXiv:2609.06581v1 Announce Type: cross
Abstract: Small-object detection remains challenging because limited pixels cause information loss and suppress the scale knowledge encoded in pretrained detec...
By Zhaoning Shi, Bo Ma