arXiv Computer Vision

HANCLIP: A Family of Hyperbolic Angular Negation Vision Language Models

arXiv Computer Vision

1d ago

HyperCLIP++: Fine-tuning CLIP forOpen-vocabulary Semantic Segmentation in Hyperbolic Space

arXiv:2609.24564v1 Announce Type: new Abstract: CLIP, a foundational vision-language model, has emerged as a powerful tool for open-vocabulary semantic segmentation. While freezing CLIP's text encode...

By Zelin Peng, Zhengqin Xu, Changsong Wen, Yu Huang, Yaoming Wang, Xiaokang Yang, Wei Shen

llms rag computer-vision fine-tuning multimodal benchmarks safety

arXiv AI

Sep 1

Hyper3-CLIP: Hierarchy-Conditioned Hyperbolic Vision-Language Training

arXiv:2608.29313v1 Announce Type: cross Abstract: CLIP-like vision-language models (VLMs) trained with contrastive objectives learn strong global image-text representations, but their Euclidean embed...

By Matin Mahmood, Antonio Rueda-Toicen, Mohamed ElBassat, Seifeldin Elkerdany, Weixing Wang, Gerard de Melo

llms rag multimodal safety

arXiv AI

Jul 28

What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features

arXiv:2607. 23271v1 Announce Type: cross Abstract: Contrastive vision-language models such as CLIP map semantically opposite phrases (e.

By Chen-Yi Lu, Yueh-Shao Chen, Somali Chaterji

llms rag diffusion fine-tuning multimodal safety

arXiv Machine Learning

Jun 30

Contrastive vision-language learning with paraphrasing and negation

arXiv:2511. 16527v2 Announce Type: replace-cross Abstract: Contrastive vision-language models continue to be the dominant approach for image-text retrieval.

By Kwun Ho Ngan, Saman Sadeghi Afgeh, Joe Townsend, Artur d'Avila Garcez

llms rag diffusion computer-vision multimodal

arXiv AI

Jul 14

Uncertainty-guided Compositional Alignment with Part-to-Whole Semantic Representativeness in Hyperbolic Vision-Language Models

arXiv:2603. 22042v3 Announce Type: replace-cross Abstract: While Vision-Language Models (VLMs) have achieved remarkable performance, their Euclidean embeddings remain limited in capturing hierarchical relationships such as part-to-whole or parent-child structures, and often face challenges in multi-object compositional scenarios.

By Hayeon Kim, Ji Ha Jang, Junghun James Kim, Se Young Chun

llms rag multimodal benchmarks safety

arXiv Computer Vision

1d ago

Hierarchical Prompt Learning for Hyperbolic Vision-Language Models

arXiv:2609.24276v1 Announce Type: new Abstract: Hyperbolic vision-language models (VLMs) represent image and text features in a geometry naturally suited to hierarchy, but their adaptation to downstr...

By Andro Erdelez, Pascal Mettes, Behzad Bozorgtabar

llms rag multimodal benchmarks

arXiv Computer Vision

Sep 1

What is the Right Embedding Space for Contrastive Learning in Referring Expression Counting?

arXiv:2505.22850v3 Announce Type: replace Abstract: Referring Expression Counting (REC) requires distinguishing visually similar objects described by fine-grained text cues. Existing methods tackle t...

By Kostas Triaridis, Panagiotis Kaliosis, E-Ro Nguyen, Jingyi Xu, Dimitris Samaras, Hieu Le

rag multimodal benchmarks safety

arXiv Machine Learning

Jul 31

Continual Learning with Vision-Language Models via Semantic-Geometry Preservation

arXiv:2603. 12055v3 Announce Type: replace-cross Abstract: Continual learning of pretrained vision-language models (VLMs) is prone to catastrophic forgetting, yet current approaches adapt to new tasks without explicitly preserving the cross-modal semantic geometry inherited from pretraining and previous stages, allowing new-task supervision to induce geometric distortion.

By Chiyuan He, Zihuan Qiu, Fanman Meng, Runtong Zhang, Linfeng Xu, Qingbo Wu, Hongliang Li

llms efficiency multimodal benchmarks safety

arXiv Machine Learning

1d ago

Seeing Through Conflicts: Improving Instruction Hierarchy Alignment in Vision-Language Models

arXiv:2609.22234v1 Announce Type: cross Abstract: Instruction hierarchy (IH) alignment teaches language models to prioritize higher-level instructions when inputs conflict. While studied primarily in...

By Nicholas Sansoterra, Zishuo Zheng, Sachin Kumar

llms agents reinforcement-learning multimodal safety

arXiv AI

Jul 7

Text as Partial Constraint: Core-Residual Alignment for Robust Vision-Language Learning

arXiv:2607. 03143v1 Announce Type: cross Abstract: Vision-language alignment powers open-vocabulary recognition, retrieval, and LVLM grounding, yet natural captions are often underspecified, making similarity brittle and overly confident under paraphrase and omitted details.

By Chengzhen Yu, Canran Xiao, Siyuan Ma, Yang Liu

multimodal safety

arXiv AI

Jun 2

Hyperbolic and Evidence-Prioritized Experts for Large Vision-Language Models

arXiv:2606. 00275v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) have demonstrated impressive performance on multimodal tasks through scaled architectures and extensive training.

By Zijie Zhou, Dandan Zhu, Hangxiangpan Wang, Heng Zhang, Huishen Jiao, Yi Zhao

llms multimodal safety

arXiv Computer Vision

Sep 3

Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checker

The paper introduces OpenRef, a benchmark for Referring Expression Comprehension (REC) designed for open‑world scenarios. OpenRef expands beyond simple settings by including diverse visual domains, variable target counts (multi‑target and none‑target), and a rich vocabulary with proper nouns, polysemous words, and ordinal terms. It also proposes new evaluation metrics—F1 for grounding accuracy and N3R for negative expression rejection—and presents a training‑free Multi‑task Consistency Checker (MCC) that improves model performance with a single click.

By Zongjian Wu, Lei Zhang

llms robotics multimodal benchmarks

Related stories