The paper introduces a self‑supervised framework that maps text, audio, image, and video into a shared 256‑dimensional embedding space and uses iterative clustering to uncover aesthetic structure. It examines how AI’s cluster assignments diverge from human affective labels on a weakly supervised multimodal dataset. The study highlights implications for cross‑modal similarity, media organization for Retrieval‑Augmented Generation, and automated data labeling.
By Corey D. C. Heath
The paper explores how AI can develop its own aesthetic categorization of art across text, audio, image, and video without explicit labels. Using a self‑supervised framework, the authors embed these modalities into a shared 256‑dimensional space and iteratively cluster the data to uncover aesthetic structure. They compare the AI’s cluster assignments with human affective labels, highlighting divergences and discussing implications for cross‑modal similarity, media organization, and automated labeling.
Abstract4D is the largest dataset of abstract paintings, containing over 120,000 images with rich metadata and multi‑dimensional prompts that capture perceptual attributes such as form, color, texture, and composition. The dataset is annotated via a hybrid human–VLM pipeline to ensure quality and consistency. Using Abstract4D, the authors analyze the semantic structure of abstract art through large‑scale embedding visualization and establish benchmark tasks for classification, cross‑modal retrieval, and text‑to‑image generation to evaluate AI models’ perception and reproduction of abstract visual language.
By Haowei Zhang, Yuanpei Zhao, Ji-Zhe Zhou, Mao Li
arXiv:2412.00176v4 Announce Type: replace
Abstract: We explore whether pre-training on datasets with paintings is necessary for a model to learn an artistic style with only a few examples. To investi...
By Hui Ren, Joanna Materzynska, Rohit Gandikota, Giannis Daras, David Bau, Antonio Torralba
arXiv:2608.29644v1 Announce Type: cross
Abstract: Attributing an artwork to an artist has traditionally relied on detailed visual observations and descriptions, known as stylistic analysis in art his...
By Marc S. Walton, Astrid Harth
CS-CLIP is a vision‑language model that improves compositional reasoning by using scene graphs to identify compositional elements and create structured negative examples through selective masking. The approach retains only the most contradictory negatives, encouraging the model to depend on compositional structure instead of surface cues. CS-CLIP achieves state‑of‑the‑art performance on compositional reasoning benchmarks while maintaining strong cross‑modal retrieval and downstream visual reasoning capabilities with fewer training samples.
By SeongJun Jeong, Minjoon Jung, Woo Suk Choi, Youwon Jang, Byoung-Tak Zhang