arXiv AI
Jul 3
CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
arXiv:2510. 20091v3 Announce Type: replace-cross Abstract: Creativity is often seen as a hallmark of human intelligence.
The Limits of Automatic Evaluation of Creativity in Large Language Models
The paper examines whether existing automatic methods can reliably assess creativity in text produced by large language models (LLMs). By collecting human ratings on 11 creativity dimensions for both human and AI short stories, the authors compare these judgments with automated metrics and LLM-as-a-Judge evaluations. The results show a significant misalignment: automated metrics and LLM judges favor AI-generated stories and show near-zero correlation with human assessments, revealing fundamental limitations in current computational approaches to evaluating creative text.
Related stories
arXiv AI
Jun 11
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
arXiv:2606. 11762v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential.
Hugging Face Trending Papers
Jun 10
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential. Realizing this potential requires systematic and scalable methods for evaluating creativity across diverse tasks.
arXiv AI
Jun 17
Do Large Language Models Always Tell The Same Stories?
arXiv:2606. 17350v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have enabled the generation of high-quality prose, yet the question of whether these models are capable of generating diverse outputs remains contested.
arXiv AI
Aug 21
Are LLMs becoming similarly creative? Evidence from three years of models
arXiv:2608. 19437v1 Announce Type: cross Abstract: Many benchmarks track Large Language Model (LLM) performance on tasks with verifiable answers, but less is known about how LLM performance is evolving on open-ended tasks, where creativity, originality and diversity may matter as much as quality.
arXiv AI
Aug 10
CreativeInstruct: Scalably Teaching LLMs to Balance Quality, Creativity, and Diversity
arXiv:2608. 07460v1 Announce Type: cross Abstract: While post-training improves the capabilities of large language models (LLMs), it generally lowers their output diversity and creativity, negatively impacting tasks that explicitly require creativity (e.
arXiv AI
Aug 10
Recipes for Creativity: Iterative Generation and Evaluation in Large Language Models
arXiv:2608. 07243v1 Announce Type: new Abstract: Generative models are often evaluated through singular artifacts, whereas human creativity typically emerges through iterative generation, appraisal, and refinement.
arXiv AI
Jun 30
The Human Creativity Benchmark
arXiv:2606. 30561v1 Announce Type: new Abstract: Modern AI evaluation frameworks treat evaluator disagreement as noise to be resolved.
arXiv AI
3d ago
LITERARYBIGFIVE: Author-Personalized Text Generation in a Unified Interpretable Space
arXiv:2608.23124v1 Announce Type: cross Abstract: Personalized text generation for authors and literary writing is essential for applications such as adaptive writing assistants, creative support too...
arXiv AI
Jun 8
DEFINED: A Data-Efficient Computational Framework for Fine-Grained Creativity Assessment in Debate Scenarios
arXiv:2606. 07226v1 Announce Type: cross Abstract: Human creativity has emerged as a critical competency in the era of large language models.
arXiv AI
Jul 22
PlotTwist: A Creative Plot Generation Framework with Small Language Models
arXiv:2603. 16410v2 Announce Type: replace-cross Abstract: Creative plot generation presents a fundamental challenge for language models: transforming a concise premise into a coherent narrative that sustains global coherence, character development, pacing, tone consistency, and emotional progression.
arXiv AI
Aug 14
LigBench: A Unified and Human-Aligned Benchmark for LLM-based Research Idea Generation
arXiv:2608. 13136v1 Announce Type: cross Abstract: With the rapid advancement of large language models (LLMs), research idea generation has attracted increasing attention.