The paper proposes a method to improve open‑vocabulary semantic segmentation by refining the CLIP visual encoder. It introduces a selective attention mechanism that preserves regional semantics and adds visual prototypes—auxiliary memory‑based classifiers built from confidently predicted patches—to bridge the modality gap. An optimizable feature extractor with exponential moving average updates further enhances prediction quality, and experiments on five datasets confirm the approach’s effectiveness.
arXiv:2502. 06818v4 Announce Type: replace Abstract: Recent works modify CLIP to perform open-vocabulary semantic segmentation in a training-free manner (TF-OVSS).
arXiv:2607. 26107v1 Announce Type: cross Abstract: Dense vision-language understanding, including object localization, region recognition, and open-vocabulary semantic segmentation, requires associating language concepts with spatially grounded visual regions.
arXiv:2503. 15639v2 Announce Type: replace-cross Abstract: Modern scene text recognition systems often depend on large end-to-end architectures that require extensive training and are prohibitively expensive for real-time scenarios.
arXiv:2509.22650v3 Announce Type: replace Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models,...
Contrastive Language-Image Pre-training (CLIP) has been shown to have limitations in its fine-grained dense feature representation, due to its pre-training focusing on matching the whole image to a text description. Considering the large data and computational burden in pre-training a vision-language model from scratch, a series of works aim to enhance the fine-grained ability of CLIP through a fine-tuning scheme.
Textual descriptions can reduce ambiguity in medical image segmentation by specifying the finding and location to be delineated. Existing text-guided methods mainly improve where image and language fe...
MARS-CLIP is a zero‑shot semantic segmentation framework that builds on CLIP by adding a multi‑resolution feature extraction module and an attention refinement mechanism. The multi‑resolution module fuses fine‑grained local features with global context to mitigate low spatial resolution, while the attention refinement injects spatial and color biases from intermediate layers into the final self‑attention block to better recover object boundaries. Experiments on six public datasets show that MARS‑CLIP outperforms state‑of‑the‑art methods.
The paper introduces Text-to-Seed (T2S), a training‑free framework for open‑vocabulary semantic segmentation that repurposes Stable Diffusion to generate attention‑based seed points from text queries. These sparse seeds serve as point prompts for the Segment Anything Model (SAM), enabling reliable region expansion without relying on inaccurate coarse masks. T2S achieves strong performance on standard OVSS benchmarks using only the text‑to‑region correspondence of diffusion models and no task‑specific training or extra annotations.
FocusOVD is an open‑vocabulary object detector that improves small‑object detection by refining geometric and contextual features without altering the frozen vision‑language model backbone. It introduces a Reweighted Cross‑scale Feature Fusion module to balance shallow and deep features, a Residual Pinwheel‑shaped Gated Convolution module to enhance local geometry in high‑resolution features, and a Residual Convolutional Gated Linear Unit module to suppress background activations and boost region‑of‑interest discriminability. Experiments on COCO‑OVD and LVIS‑OVD show that FocusOVD outperforms state‑of‑the‑art methods, achieving a novel‑category AP of 36.3% on COCO‑OVD and an overall AP of 33.7% on LVIS‑OVD.
Crane is a CLIP‑based framework for zero‑shot anomaly detection that enhances dense localization by adapting the vision encoder with a correlation‑based attention module and conditioning learnable prompts on global image context. It further fuses anomaly‑relevant patch features into the global representation for more sensitive image‑level detection, and a variant called Crane+ leverages DINOv2 spatial correlations for stronger pixel‑level performance. Across seven industrial benchmarks, Crane raises mean image‑level AP by 4.5% and Crane+ boosts mean pixel‑level AUPRO by 9.0%.
Large-scale Vision-Language Models like CLIP have demonstrated impressive open-set localization capabilities at the image level. However, adapting this capability to pixel-level dense prediction poses challenges due to global feature biases.
arXiv:2609.22834v1 Announce Type: new Abstract: Open-vocabulary semantic segmentation (OVS) of remote sensing imagery is a challenging pixel-level task requiring strong generalization and adaptation...