arXiv Computer Vision

ViCo-SAM3: Vision-Conditioned Alignment for Open-Vocabulary Camouflaged Object Segmentation

ViCo-SAM3 introduces a Vision-Conditioned alignment framework for open-vocabulary camouflaged object segmentation. The approach adds a vision-conditioned (ViCo) module that dynamically adjusts text embeddings based on global visual context, and a vision-conditioned cross-modal binding (ViCoBind) module to improve interaction between visual and textual representations. These innovations close the semantic gap between text and pixel-level cues, enabling state‑of‑the‑art performance on the OVCamo benchmark without heavy parameter overhead.

Hugging Face Trending Papers

Jul 15

Fine-grained CLIP fine-tuning with self-annotated region alignment

Contrastive Language-Image Pre-training (CLIP) has been shown to have limitations in its fine-grained dense feature representation, due to its pre-training focusing on matching the whole image to a text description. Considering the large data and computational burden in pre-training a vision-language model from scratch, a series of works aim to enhance the fine-grained ability of CLIP through a fine-tuning scheme.

llms fine-tuning multimodal safety

arXiv Computer Vision

Sep 22

HyperCLIP++: Fine-tuning CLIP forOpen-vocabulary Semantic Segmentation in Hyperbolic Space

arXiv:2609.24564v1 Announce Type: new Abstract: CLIP, a foundational vision-language model, has emerged as a powerful tool for open-vocabulary semantic segmentation. While freezing CLIP's text encode...

By Zelin Peng, Zhengqin Xu, Changsong Wen, Yu Huang, Yaoming Wang, Xiaokang Yang, Wei Shen

llms rag computer-vision fine-tuning multimodal benchmarks safety

arXiv AI

Jun 3

Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Models

arXiv:2605. 18160v2 Announce Type: replace-cross Abstract: In recent years, multimodal large language models (MLLMs) have achieved remarkable progress, primarily attributed to effective paradigms for integrating visual and textual information.

By Xinpeng Dong, Min Zhang, Kairong Han, Xu Tan, Fei Wu, Kun Kuang

llms computer-vision multimodal benchmarks safety

arXiv AI

Jun 24

MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment

arXiv:2509. 14001v5 Announce Type: replace-cross Abstract: Personalized object detection aims to adapt a general-purpose detector to recognize user-specific instances from only a few examples.

By Elena Camuffo, Francesco Barbato, Mete Ozay, Simone Milani, Umberto Michieli

llms rag computer-vision efficiency multimodal benchmarks safety

arXiv AI

Jun 12

Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality

arXiv:2606. 13288v1 Announce Type: cross Abstract: Contrastively trained vision-language models like CLIP, have made remarkable progress in learning joint image-text representations, but still face challenges in compositional understanding.

By Wei Li, Zhen Huang, Xinmei Tian

llms diffusion multimodal benchmarks

arXiv AI

Aug 11

GRASP: Granularity-Aware Region Alignment and Semantic Prototype Learning for Fine-Grained Cross-Modal Understanding in Drone Views

arXiv:2608. 09270v1 Announce Type: cross Abstract: Fine-grained cross-modal understanding in drone views is essential for aerial vision-language navigation.

By Jiahui Cui, Yan Zhao, Kan Wei, Enze Zhu, Peirong Zhang, Lei Wang, Yiru Wang

robotics multimodal benchmarks safety

arXiv AI

Jul 3

SEPS: Semantic-enhanced Patch Slimming Framework for fine-grained cross-modal alignment

arXiv:2511. 01390v2 Announce Type: replace-cross Abstract: Fine-grained cross-modal alignment aims to establish precise local correspondences between vision and language, forming a cornerstone for visual question answering and related multimodal applications.

By Xinyu Mao, Junsi Li, Haoji Zhang, Yu Liang, Ming Sun

llms diffusion nlp multimodal safety

arXiv AI

Jul 24

DINOde: Continuous Vision-Text Alignment for Open-Vocabulary Semantic Segmentation

arXiv:2607. 21371v1 Announce Type: cross Abstract: Open-vocabulary semantic segmentation (OVSS) leverages textual semantics to segment objects beyond predefined categories.

By Sung-Hoon Yoon, Hoyong Kwon, Changgyoon Oh, Kuk-Jin Yoon

rag computer-vision multimodal benchmarks safety

arXiv Machine Learning

Jul 2

MoVA: Learning Asymmetric Dual Projections for Modular Long Video-Text Alignment

arXiv:2607. 00858v1 Announce Type: cross Abstract: Contrastive pre-training has propelled video-text alignment, yet models often inherit the critical limitations of their image-text predecessors like CLIP, resulting in entangled representations.

By Peiyuan Zhu, Shaoan Xie, Zijian Li, Yifan Shen, Namrata Deka, Harsh Shrivastava, Guangyi Chen, Kun Zhang

multimodal safety

arXiv Computer Vision

Aug 25

Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos

arXiv:2501.04001v4 Announce Type: replace Abstract: This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-...

By Haobo Yuan, Xiangtai Li, Tao Zhang, Yueyi Sun, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, Ming-Hsuan Yang

llms computer-vision fine-tuning multimodal benchmarks

arXiv AI

Aug 13

Learning from Multimodal Pseudo-Labels for Robust Open-Vocabulary Instance and Panoptic Segmentation

arXiv:2608. 11681v1 Announce Type: cross Abstract: This work addresses the challenge of open-vocabulary instance segmentation (OVIS) and open-set panoptic segmentation (OSPS), which aim to recognize both predefined and unseen object categories without exhaustive human annotations.

By Duy Tran Thanh, Yeejin Lee, Byeongkeun Kang

llms computer-vision multimodal benchmarks safety

arXiv AI

Jun 30

Steerable Visual Representations

arXiv:2604. 02327v2 Announce Type: replace-cross Abstract: Pretrained Vision Transformers (ViTs) such as DINOv2 and MAE provide generic image features that can be applied to a variety of downstream tasks such as retrieval, classification, and segmentation.

By Jona Ruthardt, Manu Gaur, Deva Ramanan, Makarand Tapaswi, Yuki M. Asano

llms computer-vision multimodal benchmarks

Related stories