The paper introduces CFM, a language‑aligned concept foundation model for vision that generates fine‑grained, human‑interpretable concepts with spatial grounding. By pairing CFM with a strong semantic foundation model, it provides explanations for downstream tasks such as classification, segmentation, and captioning. The authors also analyze local co‑occurrence of concepts to define relationships, improving concept naming and yielding richer explanations while maintaining competitive performance.
By Kai Wittenmayer, Sukrut Rao, Amin Parchami-Araghi, Bernt Schiele, Jonas Fischer
The paper investigates how Vision Transformers (ViTs) ground abstract concepts such as "angry" when training data provide limited direct evidence. It proposes a metonymic grounding mechanism where abstract predictions are driven by concrete, interpretable anchor concepts like "fire" that bridge visual signals to abstract semantics. Using Transcoders on CLIP and DINO encoders, the authors recover intermediate features linked to concrete concepts and trace their contributions in circuits that recognize abstract concepts, revealing structured metonymic circuits and distinct perceptual-to-textual routes for images with rendered text.
By Jing Ding, Ziqiao Ma, Jiayuan Mao, Joyce Chai, Freda Shi
arXiv:2608. 07886v1 Announce Type: cross Abstract: Vision-language grounding connects language to visual content, yet most existing formulations reduce grounding to a unidirectional localization problem: given a prespecified text phrase or category name, identify the corresponding image region.
By Jieyu Zhang, Ziqi Gao, Luke Zettlemoyer, Ranjay Krishna
Object hallucination in multimodal large language models arises when language priors and corpus co-occurrence bias outweigh the visual evidence, with nothing tying an individual object mention to what the image shows. Most remedies intervene at decoding time without training, yet under a unified protocol their benefit is confined to short captions;supervised fine-tuning (SFT) on a detail- rich corpus lengthens captions, but over forty percent still name absent objects.
The paper introduces the Novel Visual References Dataset (NVRD), comprising 19,176 images of 90 new visual concepts with up to 20 perturbed versions each, designed to test how vision‑language models and humans map novel visual references to language. The authors evaluate three open‑source and two closed‑source VLMs against 2,400 human judgments, finding that models struggle to learn new concepts that contradict pre‑training knowledge and tend to overgeneralize labels to stimuli humans reject. NVRD is released as a benchmark for studying visual concept learning in both humans and machines.
By Ada Defne T\"ur, Gaurav Kamath, Joyce Chai, Siva Reddy, Benno Krojer
arXiv:2607. 07179v1 Announce Type: cross Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents.
By Miguel Lopez-Duran, Elena Marrero, Julian Fierrez, Marta Robledo-Moreno, Ruben Vera-Rodriguez, Daniel DeAlcala, Aythami Morales, Ruben Tolosana, Oscar Delgado, Alvaro Ortigosa, Javier Ortega-Garcia
arXiv:2610.20083v1 Announce Type: new
Abstract: Visual world knowledge is a core capability of multimodal large language models (MLLMs), underpinning knowledge-intensive visual question answering and...
By Chen Chen, Zheming Liang, Jiaqi Wang
arXiv:2610.34610v1 Announce Type: new
Abstract: Vision-language anomaly detectors describe normal and abnormal appearance with text, and their gains are commonly credited to the semantic content of t...
By Arda Baris Basaran, Mesut Ceylan, Elgar Fleisch, Filipe Barata
arXiv:2608.30621v1 Announce Type: cross
Abstract: Collecting natural-language referring expressions along with region annotations, such as masks or boxes, is a major bottleneck in visual grounding (V...
By Junbeom Hong, Seonghoon Yu, Hyung Rok Jung, Sundong Kim, Jeany Son
arXiv:2606. 29069v1 Announce Type: new Abstract: Concept-based Explainable AI (C-XAI) seeks human-understandable explanations grounded in semantic concepts, yet validation is limited by the scarcity of fine-grained concept annotations.
By Darian Fern\'andez-Guti\'errez, Rafael Bello, Marilyn Bello, Natalia D\'iaz-Rodr\'iguez
Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents. Although Vision-Language Models (VLMs) have shown remarkable performance in text-vision tasks, their robustness and transferability to different document domains remains underexplored.
arXiv:2610.08401v1 Announce Type: cross
Abstract: While recent vision-language models (VLMs) have shown outstanding performance across diverse applications, they tend to under-use visual information...
By Seulgi Kim, Zhixiong Zhang, Xinwei Zhang, Jie Ling, Ronn Shaw