The paper introduces Design Creativity Bench, a benchmark for evaluating creativity in large language model (LLM)-generated user interface (UI) designs. It measures originality (distinctiveness among models on the same prompt), creative range (how designs vary across prompts for the same UI goal), and appropriateness (how many brief criteria each design meets). Results show LLMs produce designs that are generally appropriate but far less original and varied than human designs, indicating a need for stronger measures to address this limitation.
By Aman Rusia, Abhijit Bhole, Prashank Gupta, Dipanjan Dey
arXiv:2608. 19437v1 Announce Type: cross Abstract: Many benchmarks track Large Language Model (LLM) performance on tasks with verifiable answers, but less is known about how LLM performance is evolving on open-ended tasks, where creativity, originality and diversity may matter as much as quality.
By Nirav Patel, Josiah Crossman, Eva Aggarwal, Emily Wenger
The paper introduces persona diversification as a set‑level conditioning strategy to reduce homogeneity in large language model outputs. It explores two design axes—selecting versus generating personas and space‑filling versus frontier‑seeking diversity—implementing four methods that span coverage and dispersion subset selections, uniform‑coverage sampling, and evolutionary persona generation. Experiments on tasks such as the Alternative Uses Task, Infinity‑Chat, and Divergent Association Task demonstrate significant gains in response diversity, originality, flexibility, and overall creativity, while maintaining high validity.
By Sang Bin Moon, Nicole Cho, Daniel Borrajo, Sumitra Ganesh, Abolfazl Hashemi
arXiv:2608. 07243v1 Announce Type: new Abstract: Generative models are often evaluated through singular artifacts, whereas human creativity typically emerges through iterative generation, appraisal, and refinement.
By Rens Anderson, Tessa Verhoef, Amirhossein Zohrehvand
arXiv:2606. 30561v1 Announce Type: new Abstract: Modern AI evaluation frameworks treat evaluator disagreement as noise to be resolved.
By Aspen Hopkins, Allison Nulty, Alexandria Minetti, Anoop Pakki, Angad Singh
arXiv:2510. 20091v3 Announce Type: replace-cross Abstract: Creativity is often seen as a hallmark of human intelligence.
By Zhaoyi Joey Hou, Bowei Alvin Zhang, Yining Lu, Bhiman Kumar Baghel, Anneliese Brei, Ximing Lu, Meng Jiang, Faeze Brahman, Snigdha Chaturvedi, Haw-Shiuan Chang, Daniel Khashabi, Xiang Lorraine Li
arXiv:2603. 19087v2 Announce Type: replace Abstract: Creativity is the ability to come up with novel ideas, a capacity crucial for human development and flourishing.
By Qiawen Ella Liu, Marina Dubova, Henry Conklin, Takumi Harada, Thomas L. Griffiths
The paper proposes that creativity in large language models can be measured by performance on the long tail of a reference input-output distribution, using n‑gram absence from pretraining data as a novelty proxy. Across 15 benchmarks and 26 open‑weight models, the authors find that creativity benchmarks occupy further tail regions than standard benchmarks and that higher‑tail subsets better align with human creativity judgments. They introduce a new diagnostic, q@Tail(α), to assess mean task quality on the tail fraction of existing benchmarks, revealing significant potential for improving LLM creativity through stronger tail performance.
By Yichen Wang, Tenghao Huang, Vishakh Padmakumar, Mina Lee
arXiv:2606. 11762v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential.
By Min Sen Tan, Zachary Kit Chun Choy, Syed Ali Redha Alsagoff, Nadya Yuki Wangsajaya, Mohor Banerjee, Swaagat Bikash Saikia, Alvin Chan
The paper introduces a new inference architecture for vibe design agents that separates design exploration from implementation. By generating structured design specifications with typicality scores and selecting one for downstream generation, the method allows users to explore coherent UI alternatives without altering the underlying generation settings. Experiments on UI themes and visual-asset prompts show increased selection coverage and screenshot variation, with mixed preferences from an LLM judge and modest operational costs in a large online test.
By Yifan Zhang, Nghi D. Q. Bui, Georgios Evangelopoulos, Arnaud Benard
Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential. Realizing this potential requires systematic and scalable methods for evaluating creativity across diverse tasks.
The paper introduces CreaEval, an automated creativity evaluator designed for complex multi-step tasks (CGPST). It separates the evaluation process into two phases: a memory‑augmented analysis that transforms responses into structured evidence, and an evidence‑based judging step that scores without seeing raw outputs. Experiments show CreaEval outperforms existing baselines by an average of 22.74% across CGPST and two simpler creativity tasks.
By Xiangyu Wang, Jin Wu, Xiaoyu Li, Chanjin Zheng, Yifeng Zhou