The paper proposes augmenting the global [CLS] token and register tokens of vision transformers with spatial region tokens to better capture fine‑grained image details. By matching each semantic token with a corresponding image patch, the method automatically generates localized ROI tokens without external annotations. These tokens are then used in a multi‑vector retrieval framework inspired by ColBERT, improving fine‑grained image retrieval while keeping storage costs manageable.
arXiv:2607. 04605v1 Announce Type: cross Abstract: Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive.
Multi-vector vision-language retrieval preserves fine-grained visual evidence through maximum-similarity late interaction, but dense image-side tokens make storage and scoring expensive. Existing token compression methods reduce this cost, yet they can remove or collapse object- and region-level evidence that future query tokens may need to select.
arXiv:2607.09086v2 Announce Type: replace Abstract: We present Subtoken Vision Transformer (SubViT), a selective image tokenization method for fine-grained visual recognition. Standard Vision Transfo...
MINER is a training‑free inference framework that enhances frozen dual‑encoder models for text‑to‑image retrieval when queries refer to small, visually subordinate objects in cluttered scenes. It augments the global image embedding with a bank of region‑level embeddings and applies hubness‑correcting similarity rescoring, thereby recovering visual evidence that global pooling underweights. The authors introduce ROCS, a benchmark derived from Flickr30K and MS COCO, and demonstrate that MINER improves retrieval performance across CLIP, SigLIP, and SigLIP 2 backbones on both ROCS and standard splits, attributing gains mainly to broader spatial coverage rather than precise crop placement.
arXiv:2609.36560v1 Announce Type: cross Abstract: Object re-identification (ReID) faces a recurring challenge: different identities can share highly similar global appearances, while the cues that di...
Object re-identification (ReID) faces a recurring challenge: different identities can share highly similar global appearances, while the cues that distinguish them are localized, heterogeneous, and vi...
Vision-language models commonly project all tokens produced by a pretrained vision encoder into a large language model. However, final-layer features can discard text, local attributes, and spatial relationships, while high-resolution inputs substantially increase context length and inference latency.
The paper introduces CERES, a closed‑loop multimodal indexing framework that addresses semantic collapse in multimodal generation by building a three‑level semantic pyramid and using scale‑routed cross‑attention to generate images that remain retrievable by their original queries. CERES employs a co‑occurrence‑aware router, a lightweight U‑Net generator, and a soft‑Jaccard coverage objective to ensure generated images cover the intended concepts, verified by re‑indexing with a frozen vision‑language model and an external DINOv2 probe. Experiments on four pansharpening benchmarks show state‑of‑the‑art performance, especially under extreme scale variation, and significant improvements in concept‑query retrieval and image‑text ranking metrics.
The paper introduces RVSD, a training‑free, plug‑and‑play decoding framework that combines token sparsification with Semantic‑Space Visual Retrieval (SSVR) to reduce visual hallucinations in large vision‑language models. RVSD employs a semantics‑directed token selection strategy to prune redundant tokens while preserving essential visual information, and uses SSVR to perform on‑demand cross‑modal retrieval within a shared semantic space. Experiments show that RVSD achieves state‑of‑the‑art performance in mitigating visual hallucinations while maintaining strong suppression in long‑context generation.
GeoStore is a new benchmark for fine‑grained point‑of‑interest (POI) localization that matches close‑up storefront photos against large geo‑tagged street‑view images, a task distinct from traditional visual place recognition. The paper shows that global‑descriptor methods designed for symmetric matching perform poorly on this asymmetric problem, and introduces GLAM, a Global‑to‑Local Asymmetric Matching approach that combines a global retrieval anchor with a lightweight local re‑ranking using pooled region tokens. GLAM achieves higher Recall@1/5/10 and mAP than strong baselines while using far fewer re‑ranking features and significantly lower per‑pair matching cost.
arXiv:2607. 08541v1 Announce Type: cross Abstract: Open-vocabulary object detection and segmentation aim to recognize arbitrary objects beyond predefined categories.
arXiv:2609.36916v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) incur high inference latency from long visual token sequences. Existing pruning methods commonly use attention...