arXiv:2608. 15828v1 Announce Type: cross Abstract: Current evaluation of metaphor explanations relies mainly on holistic quality ratings, revealing little about how explanation quality is structured or where human judgments agree and diverge.
arXiv:2510. 04120v2 Announce Type: replace-cross Abstract: Large language models (LLMs) achieve strong performance on metaphor detection and interpretation tasks, yet it remains unclear what such behavioral success reveals about metaphor processing.
The paper introduces VMetaphor-Bench, a benchmark for evaluating visual metaphor generation in text-to-image models, comprising 1,500 curated metaphors across three levels and ten categories, each paired with two prompts of varying specificity. It proposes a hybrid evaluation framework using a multiple-choice question protocol and dimension-based scoring to assess metaphorical fidelity. Experiments on 11 T2I models show that even top proprietary models struggle with compositional structuring and cross-domain mapping, underscoring the need for further research in this area.
The paper introduces VMetaphor-Bench, a benchmark for assessing visual metaphor generation in text-to-image models. It contains 1,500 curated metaphors across three levels and ten categories, each paired with two prompts of varying specificity. The authors evaluate 11 T2I models using a hybrid MLLM-as-judge framework that combines a large multiple-choice question set with dimension-based scoring, finding that even top proprietary models struggle with compositional structuring and cross-domain mapping.
arXiv:2601. 03388v3 Announce Type: replace-cross Abstract: Earlier research has shown that metaphors influence human decision-making, raising the question of whether metaphors also influence large language models (LLMs)' reasoning pathways, given that their training data contain a large number of metaphors.
MetaHOPE is an error‑severity‑aware annotation framework designed to evaluate how well machine translation (MT) and large language models (LLMs) translate metaphors. The authors applied MetaHOPE to three state‑of‑the‑art systems—GoogleMT, GPT5.4, and Hunyuan‑7b—using two human‑annotated metaphor corpora (VUAMC and PSUCMC) for English‑to‑Chinese and Chinese‑to‑English translation. They also produced a bilingual post‑edited gold reference, creating a new resource for metaphor translation research.
Similes provide a compact and expressive way to describe visual characteristics in text prompts. Recent text-to-image models (t2i models) can produce visually compelling outputs from simile prompts, yet even frontier models frequently misinterpret the metaphorical vehicle and confuse it with the object.
PRISM is a modality‑agnostic, category‑theoretic framework that measures and refines multimodal analogies by representing them as explicit relational mappings. It introduces a pullback score to quantify relational alignment and an iterative refinement loop that uses this score as feedback to improve generated images. On the AnaloBench benchmark, PRISM’s pullback score alone achieves 82.5% accuracy, and human evaluations show a 57.65% preference for refined outputs, though refinement may sometimes favor visually crowded compositions.
arXiv:2602. 05493v2 Announce Type: replace-cross Abstract: Data annotation remains a significant bottleneck in the field of humanities and social sciences, particularly for complex linguistic tasks such as metaphor identification.
The paper examines whether existing automatic methods can reliably assess creativity in text produced by large language models (LLMs). By collecting human ratings on 11 creativity dimensions for both human and AI short stories, the authors compare these judgments with automated metrics and LLM-as-a-Judge evaluations. The results show a significant misalignment: automated metrics and LLM judges favor AI-generated stories and show near-zero correlation with human assessments, revealing fundamental limitations in current computational approaches to evaluating creative text.
The study investigates how multilingual training affects figurative language identification in proverbs, using 742 proverb concepts translated into seven languages. Five models—including multilingual encoders and instruction‑tuned LLMs—were evaluated with varying levels of multilingual supervision, and a new multidimensional annotation framework was introduced to classify proverbs into metaphorical, moral/advisory, cause‑effect, and culture‑specific forms. Results show that adding multilingual data beyond 50% yields limited gains, but the best supervision level depends on the model and language; combining diverse figurative forms improves overall performance, especially for the least frequent culture‑specific form.
arXiv:2608. 03095v1 Announce Type: cross Abstract: We present VIVID (Vietnamese Idioms for Validation and Interpretation Depth), the first systematic benchmark for evaluating culturally grounded figurative language understanding in Vietnamese.
The paper introduces Visual Metaphor Transfer (VMT), a task that requires models to extract the abstract ‘creative essence’ from a reference image and apply it to a new target subject. It proposes a multi‑agent framework based on Conceptual Blending Theory, using a Schema Grammar to separate relational invariants from visual entities. The system includes perception, transfer, generation, and diagnostic agents, and experimental results show it outperforms state‑of‑the‑art baselines in metaphor consistency, analogy appropriateness, and visual creativity.