arXiv Computer Vision
Seeing Before Naming: Context-Grounded Learning of Novel Visual Concepts
Read the original on arXiv Computer Vision →The Flow has not summarised this story yet — read it at arXiv Computer Vision.
Related stories
arXiv AI
Aug 24
CFM: Language-aligned Concept Foundation Model for Vision
The paper introduces CFM, a language‑aligned concept foundation model for vision that generates fine‑grained, human‑interpretable concepts with spatial grounding. By pairing CFM with a strong semantic foundation model, it provides explanations for downstream tasks such as classification, segmentation, and captioning. The authors also analyze local co‑occurrence of concepts to define relationships, improving concept naming and yielding richer explanations while maintaining competitive performance.
arXiv AI
Oct 7
Metonymic Circuits for Abstract Concept Grounding in Vision Transformers
The paper investigates how Vision Transformers (ViTs) ground abstract concepts such as "angry" when training data provide limited direct evidence. It proposes a metonymic grounding mechanism where abstract predictions are driven by concrete, interpretable anchor concepts like "fire" that bridge visual signals to abstract semantics. Using Transcoders on CLIP and DINO encoders, the authors recover intermediate features linked to concrete concepts and trace their contributions in circuits that recognize abstract concepts, revealing structured metonymic circuits and distinct perceptual-to-textual routes for images with rendered text.
arXiv AI
Aug 11
Vision-Language Grounding as Bidirectional Concept Correspondence
arXiv:2608. 07886v1 Announce Type: cross Abstract: Vision-language grounding connects language to visual content, yet most existing formulations reduce grounding to a unidirectional localization problem: given a prespecified text phrase or category name, identify the corresponding image region.
Hugging Face Trending Papers
Aug 13
Dual-Stream Cross-Anchor Correction Grounding Long-Form Captions and the Domain Limits of Object-Level Anchors
Object hallucination in multimodal large language models arises when language priors and corpus co-occurrence bias outweigh the visual evidence, with nothing tying an individual object mention to what the image shows. Most remedies intervene at decoding time without training, yet under a unified protocol their benefit is confined to short captions;supervised fine-tuning (SFT) on a detail- rich corpus lengthens captions, but over forty percent still name absent objects.
arXiv Computation and Language
Oct 19
Would you still call this Dax? Novel Visual References in VLMs and Humans
The paper introduces the Novel Visual References Dataset (NVRD), comprising 19,176 images of 90 new visual concepts with up to 20 perturbed versions each, designed to test how vision‑language models and humans map novel visual references to language. The authors evaluate three open‑source and two closed‑source VLMs against 2,400 human judgments, finding that models struggle to learn new concepts that contradict pre‑training knowledge and tend to overgeneralize labels to stimuli humans reject. NVRD is released as a benchmark for studying visual concept learning in both humans and machines.
arXiv Machine Learning
Jul 9
Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering
arXiv:2607. 07179v1 Announce Type: cross Abstract: Document Visual Question Answering (DocVQA) presents a complex multimodal challenge, requiring models to exploit visual, textual, and layout information from documents.