← Back to all news
arXiv AI By Jing Ding, Ziqiao Ma, Jiayuan Mao, Joyce Chai, Freda Shi

Metonymic Circuits for Abstract Concept Grounding in Vision Transformers

Read the original on arXiv AI →

The paper investigates how Vision Transformers (ViTs) ground abstract concepts such as "angry" when training data provide limited direct evidence. It proposes a metonymic grounding mechanism where abstract predictions are driven by concrete, interpretable anchor concepts like "fire" that bridge visual signals to abstract semantics. Using Transcoders on CLIP and DINO encoders, the authors recover intermediate features linked to concrete concepts and trace their contributions in circuits that recognize abstract concepts, revealing structured metonymic circuits and distinct perceptual-to-textual routes for images with rendered text.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Oct 9

From What to Which: Decoding Modifier Grounding in Frozen MLLMs

The paper investigates whether frozen Multimodal Large Language Models (MLLMs) encode grounding information about specific visual instances within the tokens they generate, including modifiers such as attributes, spatial expressions, and relational terms. The authors introduce OTTER, a lightweight supervised probe that applies Optimal Transport to align generated tokens with visual regions, producing compact grounding maps. Experiments demonstrate that modifier tokens, especially spatial terms, carry instance‑discriminative visual information, that object nouns also encode referential cues, and that the recovered grounding remains robust to context changes and generalizes to free generation and cross‑dataset transfer.

By Barbara Toniella Corradini (AI for Good), Caterina Gallegati (University of Siena, Italy), Ludovica Genovese (AI for Good), Vittorio Murino (AI for Good)
llmsmultimodal
More like this →