The study demonstrates that large DINOv3 vision transformers, trained solely on images without any language supervision, can organize rendered words in their embedding space according to meaning. By deriving word representations from differences between word images and matched controls, the authors show that semantically related words cluster together, brand names group by industry, and country-capital pairs exhibit consistent vector offsets. These representations align with visual object categories and share structure with text and speech encoders, indicating that image-only training can impart both visual and semantic understanding of words.
arXiv:2609.28187v1 Announce Type: new Abstract: Self-supervised vision transformers trained with DINO-style objectives exhibit striking emergent semantic representation quality across visual tasks, y...
arXiv:2607. 16214v1 Announce Type: cross Abstract: Image descriptions represented with language models (LMs) predict human brain responses to naturalistic images in high-level visual regions, but the factors driving this predictivity remain unclear.
The paper investigates how vision‑language models (VLMs) perform optical character recognition (OCR) by identifying attention heads that are causally necessary for OCR across four models. These heads are shown to be general‑purpose, producing interpretable semantic features for any image token, such as recognizing the word "bike" or the concept "feathers". By collapsing the heads’ attention weights into a verbalization lens transformation, the authors reveal that image representations align with language from early layers and can even be used to edit non‑word concepts in images, demonstrating the broader utility of this subspace.
The paper investigates how Vision Transformers (ViTs) ground abstract concepts such as "angry" when training data provide limited direct evidence. It proposes a metonymic grounding mechanism where abstract predictions are driven by concrete, interpretable anchor concepts like "fire" that bridge visual signals to abstract semantics. Using Transcoders on CLIP and DINO encoders, the authors recover intermediate features linked to concrete concepts and trace their contributions in circuits that recognize abstract concepts, revealing structured metonymic circuits and distinct perceptual-to-textual routes for images with rendered text.
arXiv:2605.00809v3 Announce Type: replace Abstract: In this paper, we present \textbf{Gen}erative \textbf{L}anguage-\textbf{I}mage \textbf{P}re-training (GenLIP), a minimalist generative pretraining...
arXiv:2606. 29416v1 Announce Type: cross Abstract: Can a vision model truly see an object, or does it only fit surface-level visual cues?
arXiv:2603. 04419v2 Announce Type: replace-cross Abstract: We characterize the phenomenon of context-dependent affordance computation in vision-language models (VLMs).
The paper investigates the effect of using semantic positive pairs—different instances of the same class—in self‑supervised visual representation learning. By creating matched ImageNet‑1K subsets of augmented pairs and manually curated semantic pairs, the authors compare contrastive and non‑contrastive SSL methods under identical training conditions. Across transfer learning and object detection tasks, semantic‑pair pretraining consistently outperforms augmented‑pair pretraining, with contrastive methods like SimCLR showing the largest gains, indicating that semantic pairs foster additional invariances beyond standard augmentations.
GraSP‑VL demonstrates that the length of frozen vision‑language embeddings can serve as a controllable semantic interface. By learning a shared near‑orthogonal prefix transform, the method creates a Semantic Matryoshka where short prefixes encode coarse semantics and longer prefixes reveal finer language‑grounded distinctions, all while preserving the original embedding geometry. Experiments on COCO/Flickr30K and SugarCrepe‑clean show strong performance with negligible drift in the full embedding space.
arXiv:2608. 07886v1 Announce Type: cross Abstract: Vision-language grounding connects language to visual content, yet most existing formulations reduce grounding to a unidirectional localization problem: given a prespecified text phrase or category name, identify the corresponding image region.
arXiv:2609.37230v1 Announce Type: cross Abstract: Visual distinctions are often finer than those reflected in linguistic conceptualization. Vision-language models exhibit a similar asymmetry: a disti...
arXiv:2503. 07265v4 Announce Type: replace-cross Abstract: Text-to-Image (T2I) models are capable of generating high-quality artistic creations and visual content.