arXiv AI

PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies

PRISM is a modality‑agnostic, category‑theoretic framework that measures and refines multimodal analogies by representing them as explicit relational mappings. It introduces a pullback score to quantify relational alignment and an iterative refinement loop that uses this score as feedback to improve generated images. On the AnaloBench benchmark, PRISM’s pullback score alone achieves 82.5% accuracy, and human evaluations show a 57.65% preference for refined outputs, though refinement may sometimes favor visually crowded compositions.

arXiv Computer Vision
Aug 31

Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning

The paper introduces Visual Metaphor Transfer (VMT), a task that requires models to extract the abstract ‘creative essence’ from a reference image and apply it to a new target subject. It proposes a multi‑agent framework based on Conceptual Blending Theory, using a Schema Grammar to separate relational invariants from visual entities. The system includes perception, transfer, generation, and diagnostic agents, and experimental results show it outperforms state‑of‑the‑art baselines in metaphor consistency, analogy appropriateness, and visual creativity.

By Yu Xu, Yuxin Zhang, Lin Gao, Oliver Deussen, Tong-Yee Lee, Fan Tang
arXiv AI
Sep 3

Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models

The paper introduces VMetaphor-Bench, a benchmark for evaluating visual metaphor generation in text-to-image models, comprising 1,500 curated metaphors across three levels and ten categories, each paired with two prompts of varying specificity. It proposes a hybrid evaluation framework using a multiple-choice question protocol and dimension-based scoring to assess metaphorical fidelity. Experiments on 11 T2I models show that even top proprietary models struggle with compositional structuring and cross-domain mapping, underscoring the need for further research in this area.

By Chuer Chen, Zichen Wang, Yi He, Zhengxi Yu, Nan Cao
arXiv Computation and Language
Sep 1

UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models

UReason is a benchmark that evaluates how well unified multimodal models (UMMs) align textual reasoning with image generation. It contains 2,000 human‑curated instances across five reasoning‑intensive tasks—Code, Arithmetic, Spatial, Attribute, and Text—and compares direct generation, reasoning‑guided generation, and decontextualized generation. The study finds that while reasoning‑guided generation improves over direct generation, decontextualized generation consistently outperforms it, indicating that the visual semantics in textual reasoning are not reliably reflected in the generated images.

By Cheng Yang, Chufan Shi, Bo Shui, Yaokang Wu, Muzi Tao, Huijuan Wang, Ivan Yee Lee, Yong Liu, Xuezhe Ma, Taylor Berg-Kirkpatrick
Hugging Face Trending Papers
Sep 2

Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models

The paper introduces VMetaphor-Bench, a benchmark for assessing visual metaphor generation in text-to-image models. It contains 1,500 curated metaphors across three levels and ten categories, each paired with two prompts of varying specificity. The authors evaluate 11 T2I models using a hybrid MLLM-as-judge framework that combines a large multiple-choice question set with dimension-based scoring, finding that even top proprietary models struggle with compositional structuring and cross-domain mapping.