arXiv Computer Vision

SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models

SOCO is a new benchmark for Semantic Object Correspondence that introduces a taxonomy of correspondence types and provides consistent, functionally meaningful keypoint annotations across 100 categories and over 1M correspondence pairs. It also includes keypoint language descriptions, enabling evaluation of large vision‑language models and their fine‑grained part‑level understanding. Experiments show that vision foundation backbones encode strong semantic structure but transfer correspondences poorly across related categories, LVLMs excel at text‑prompted part localization but lag in visual‑reference matching, and correspondence performance predicts dense downstream tasks more strongly than ImageNet classification.

arXiv AI

Aug 24

CFM: Language-aligned Concept Foundation Model for Vision

The paper introduces CFM, a language‑aligned concept foundation model for vision that generates fine‑grained, human‑interpretable concepts with spatial grounding. By pairing CFM with a strong semantic foundation model, it provides explanations for downstream tasks such as classification, segmentation, and captioning. The authors also analyze local co‑occurrence of concepts to define relationships, improving concept naming and yielding richer explanations while maintaining competitive performance.

By Kai Wittenmayer, Sukrut Rao, Amin Parchami-Araghi, Bernt Schiele, Jonas Fischer

computer-vision benchmarks

arXiv Machine Learning

6d ago

DisParQ: Self-Supervised Part Concepts for Interpretable Vision Foundation Models

arXiv:2610.09802v1 Announce Type: cross Abstract: Concept-based vision models represent images through an intermediate layer of human-inspectable concepts, so what a model relies on can be traced to...

By Adam Pardyl, Siddhartha Gairola, Sukrut Rao, Adam Wr\'obel, Bartosz Zieli\'nski, Bernt Schiele, Dawid Rymarczyk

efficiency benchmarks

Hugging Face Trending Papers

Sep 16

PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection

Intelligent systems that act in the world require image understanding that is both comprehensive and spatially grounded. Current vision-language models (VLMs) can generate fluent and detailed image ca...

llms computer-vision multimodal benchmarks

arXiv AI

Aug 11

Vision-Language Grounding as Bidirectional Concept Correspondence

arXiv:2608. 07886v1 Announce Type: cross Abstract: Vision-language grounding connects language to visual content, yet most existing formulations reduce grounding to a unidirectional localization problem: given a prespecified text phrase or category name, identify the corresponding image region.

By Jieyu Zhang, Ziqi Gao, Luke Zettlemoyer, Ranjay Krishna

llms computer-vision multimodal safety

arXiv AI

Jul 31

TraceCLIP: Recovering Local Semantics from Patch-to-CLS Contributions

arXiv:2607. 26107v1 Announce Type: cross Abstract: Dense vision-language understanding, including object localization, region recognition, and open-vocabulary semantic segmentation, requires associating language concepts with spatially grounded visual regions.

By Xinran Liu, Shouqian Shi, Yutong Chen, Ge Wang, Xin-Wei Yao, Sheng Zhong

rag computer-vision multimodal benchmarks safety

arXiv Computation and Language

Sep 17

PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection

PANORAMA introduces a new panoptic grounded captioning framework that jointly generates detailed image captions and associates each phrase with precise pixel-level masks. The authors create PanoCaps, a human‑annotated benchmark with dense captions and near‑complete pixel coverage, and propose a phrase‑mask matching protocol with a generalized Panoptic Quality metric. PANORAMA conditions a pretrained segmenter on contextualized phrase representations, learns to select appropriate masks, and achieves state‑of‑the‑art grounding performance on PanoCaps and other pixel‑level tasks.

By Sara Pieri, Evangelos Kazakos, Shizhe Chen, Josef Sivic, Cordelia Schmid

llms computer-vision multimodal benchmarks

arXiv Machine Learning

Aug 12

Cross-View Feature Matching: Survey, Benchmarking, and Foundation-Model Perspectives

arXiv:2608. 11093v1 Announce Type: new Abstract: Cross-view feature matching aims to establish reliable correspondences across images with large viewpoint variations.

By Songlin Du, Xiaoyong Lu, Zeyu Wu, Xiaobo Lu, Guobao Xiao, Bin Fan, Jiayi Ma, Takeshi Ikenaga

benchmarks

arXiv AI

Jun 24

Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations

arXiv:2606. 24716v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence.

By Jonas Klotz, Cassio F. Dantas, Pallavi Jain, Diego Marcos, Beg\"um Demir

llms rag benchmarks safety

arXiv Computer Vision

Oct 1

FAST: Flow Any Scene Transformer

arXiv:2609.39748v1 Announce Type: new Abstract: Scaling has become a primary driver of progress in language and vision foundation models, yet its role in precise correspondence matching remains under...

By Yongjian Zhang, Longguang Wang, Zhuo Song, Zhiheng Fu, Liang Lin, Yulan Guo

llms benchmarks

arXiv Machine Learning

Jun 9

Zero-Shot Semantic Re-Identification for Autonomous Driving: A VLM Baseline Study

arXiv:2606. 09362v1 Announce Type: cross Abstract: Re-Identification (ReID) in autonomous driving is typically formulated as a visual matching problem, where observations of vehicles, pedestrians, and cyclists are associated across time, frames, or camera views using learned appearance embeddings, often complemented by motion, geometric, or multimodal cues.

By Eduardo Borges, Manuel Abreu, Lu\'is Garrote, Urbano J. Nunes

llms rag agents robotics multimodal benchmarks safety

arXiv Computer Vision

Sep 4

FoRIS: Progressive Foreground Refinement for Training-Free In-Context Segmentation

FoRIS is a training‑free in‑context segmentation framework that refines foreground masks through a coarse‑to‑fine process. It operates in three stages—Foreground Purification, Localization, and Consolidation—to suppress background noise, pinpoint target regions, and reconstruct complete foreground structures. The method achieves state‑of‑the‑art performance, improving mIoU by 4.5 and 4.8 points in 1‑shot and 5‑shot settings respectively.

By Ming Hu, Jianfu Yin, Mingyu Dou, Miaomiao Zhang, Yao Wang, Cong Hu, Bingliang Hu, Quan Wang

computer-vision benchmarks

Hugging Face Trending Papers

Sep 30

FAST: Flow Any Scene Transformer

Scaling has become a primary driver of progress in language and vision foundation models, yet its role in precise correspondence matching remains underexplored. In this work, we present Flow Any Scene...

llms benchmarks

Related stories