arXiv Computer Vision October 23, 2026 By Aijing Yu, Zhengbo Wang, Lijun Sheng, Jian Liang, Xiao-Yu Zhang

Refining Open-Vocabulary Semantic Segmentation via Regional Semantics and Visual Prototypes

Read the original on arXiv Computer Vision →

The paper proposes a method to improve open‑vocabulary semantic segmentation by refining the CLIP visual encoder. It introduces a selective attention mechanism that preserves regional semantics and adds visual prototypes—auxiliary memory‑based classifiers built from confidently predicted patches—to bridge the modality gap. An optimizable feature extractor with exponential moving average updates further enhances prediction quality, and experiments on five datasets confirm the approach’s effectiveness.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computer Vision.

arXiv Machine Learning

Jul 20

Rethinking the Global Knowledge of CLIP in Training-Free Open-Vocabulary Semantic Segmentation

arXiv:2502. 06818v4 Announce Type: replace Abstract: Recent works modify CLIP to perform open-vocabulary semantic segmentation in a training-free manner (TF-OVSS).

By Jingyun Wang, Cilin Yan, Guoliang Kang

rag computer-vision benchmarks

arXiv AI

Jul 31

TraceCLIP: Recovering Local Semantics from Patch-to-CLS Contributions

arXiv:2607. 26107v1 Announce Type: cross Abstract: Dense vision-language understanding, including object localization, region recognition, and open-vocabulary semantic segmentation, requires associating language concepts with spatially grounded visual regions.

By Xinran Liu, Shouqian Shi, Yutong Chen, Ge Wang, Xin-Wei Yao, Sheng Zhong

rag computer-vision multimodal benchmarks safety

arXiv AI

Jun 2

A Lightweight Context-Driven Training-Free Network for Scene Text Segmentation and Recognition

arXiv:2503. 15639v2 Announce Type: replace-cross Abstract: Modern scene text recognition systems often depend on large end-to-end architectures that require extensive training and are prohibitively expensive for real-time scenarios.

By Ritabrata Chakraborty, Shivakumara Palaiahnakote, Umapada Pal, Cheng-Lin Liu

computer-vision benchmarks

arXiv Computer Vision

Sep 22

RefAM: Attention Magnets for Zero-Shot Referral Segmentation

arXiv:2509.22650v3 Announce Type: replace Abstract: Most existing approaches to referring segmentation achieve strong performance only through fine-tuning or by composing multiple pre-trained models,...

By Anna Kukleva, Enis Simsar, Alessio Tonioni, Muhammad Ferjad Naeem, Federico Tombari, Jan Eric Lenssen, Bernt Schiele

llms diffusion computer-vision fine-tuning multimodal benchmarks

Hugging Face Trending Papers

Jul 15

Fine-grained CLIP fine-tuning with self-annotated region alignment

Contrastive Language-Image Pre-training (CLIP) has been shown to have limitations in its fine-grained dense feature representation, due to its pre-training focusing on matching the whole image to a text description. Considering the large data and computational burden in pre-training a vision-language model from scratch, a series of works aim to enhance the fine-grained ability of CLIP through a fine-tuning scheme.

llms fine-tuning multimodal safety

Hugging Face Trending Papers

Sep 24

Multimodal Routing and Region Refinement for Language-Guided Medical Image Segmentation

Textual descriptions can reduce ambiguity in medical image segmentation by specifying the finding and location to be delineated. Existing text-guided methods mainly improve where image and language fe...

computer-vision multimodal