The paper reports a new phenomenon in CLIP embeddings where human and AI‑generated paintings naturally separate along dominant principal directions without any supervised training. The authors investigate this separation by linking embedding directions back to image features using interpretable representations and gradient‑based inversion, finding that the separation is driven by distributed multiscale image structure rather than simple global or local statistics. They also show that small, imperceptible image perturbations can cause large displacements along these directions, highlighting a mismatch between CLIP’s visual evidence and human perception.
We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP’s accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar models learn.
Composition, the deliberate arrangement of visual elements, is central to how meaning, emotion, and aesthetic quality are conveyed in artwork, yet it remains among the least formalized dimensions of visual understanding. Prior work highlights a persistent gap in learning meaningful compositional representations, attributing it to semantic bias and suggesting that human-inspired approaches may be key.
Abstract4D is the largest dataset of abstract paintings, containing over 120,000 images with rich metadata and multi‑dimensional prompts that capture perceptual attributes such as form, color, texture, and composition. The dataset is annotated via a hybrid human–VLM pipeline to ensure quality and consistency. Using Abstract4D, the authors analyze the semantic structure of abstract art through large‑scale embedding visualization and establish benchmark tasks for classification, cross‑modal retrieval, and text‑to‑image generation to evaluate AI models’ perception and reproduction of abstract visual language.
The aim of this paper is twofold. First, it investigates whether newer generative models are getting better at pastiching contemporary artworks.
arXiv:2505. 15516v3 Announce Type: replace-cross Abstract: While eXplainable AI (XAI) has advanced significantly, few methods address interpretability in embedded vector spaces where dimensions represent complex abstractions.
The paper explores whether CLIP embeddings can detect AI-generated images by using a frozen CLIP model to extract visual embeddings and training lightweight classifiers on top. On the CIFAKE benchmark, the approach achieves 95% accuracy without language reasoning, and 85% accuracy after few-shot adaptation with 20% of the data. Certain image types, such as wide-angle photographs and oil paintings, remain challenging, highlighting unexplored difficulties in AI-generated image classification.
arXiv:2606. 00121v1 Announce Type: cross Abstract: Reconstructing visual stimuli from brain recordings has been a meaningful and challenging task in brain decoding.
CRISP (Compositional Relational Invariance from Spatial Primitives) is an image‑classification framework that decomposes visual recognition into primitive elements and their relational composition. It represents these compositions with soft unary, binary, and ternary predicates over primitive locations and appearance, enabling differentiable spatial and visual alignment learned end‑to‑end. Evaluated on five DomainBed datasets covering style, provenance, and camera‑trap shifts, CRISP achieves new state‑of‑the‑art performance on both benchmarks.
arXiv:2505. 03201v4 Announce Type: replace-cross Abstract: Integrated Gradients (IG) is a widely used attribution method in explainable AI, particularly in computer vision applications where reliable feature attribution is essential.
arXiv:2512. 08854v3 Announce Type: replace-cross Abstract: It has been hypothesized that achieving the data efficiency of human visual perception requires a generative approach in which internal representations result from inverting a decoder.
arXiv:2606. 13288v1 Announce Type: cross Abstract: Contrastively trained vision-language models like CLIP, have made remarkable progress in learning joint image-text representations, but still face challenges in compositional understanding.
arXiv:2608. 14435v1 Announce Type: cross Abstract: Frozen image embeddings from models such as CLIP are increasingly used to classify paintings by art-historical style, with high reported accuracy.