arXiv AI
Jul 3
CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
arXiv:2510. 20091v3 Announce Type: replace-cross Abstract: Creativity is often seen as a hallmark of human intelligence.
Using Poly-Encoders for Computationally Efficient Automated Creativity Assessment
The paper presents a new method for automated creativity assessment that uses Poly-Encoders to achieve high accuracy while remaining computationally efficient. By fine‑tuning a Poly-Encoder on a public dataset of 18,000 human‑rated responses from the Scientific Creative Thinking Test, the authors obtain Pearson correlations up to r = 0.74, comparable to large language models. The approach relies on small pre‑trained BERT encoders, dramatically reducing computational demands and enabling potential deployment on consumer‑grade hardware.
Related stories
arXiv AI
Aug 21
Are LLMs becoming similarly creative? Evidence from three years of models
arXiv:2608. 19437v1 Announce Type: cross Abstract: Many benchmarks track Large Language Model (LLM) performance on tasks with verifiable answers, but less is known about how LLM performance is evolving on open-ended tasks, where creativity, originality and diversity may matter as much as quality.
arXiv AI
Jun 8
DEFINED: A Data-Efficient Computational Framework for Fine-Grained Creativity Assessment in Debate Scenarios
arXiv:2606. 07226v1 Announce Type: cross Abstract: Human creativity has emerged as a critical competency in the era of large language models.
Hugging Face Trending Papers
Jun 10
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential. Realizing this potential requires systematic and scalable methods for evaluating creativity across diverse tasks.
arXiv AI
Jun 11
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
arXiv:2606. 11762v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential.
arXiv AI
Aug 10
Recipes for Creativity: Iterative Generation and Evaluation in Large Language Models
arXiv:2608. 07243v1 Announce Type: new Abstract: Generative models are often evaluated through singular artifacts, whereas human creativity typically emerges through iterative generation, appraisal, and refinement.
arXiv Computation and Language
Sep 4
Decoupled Analysis-Judging: An Automated Creativity Evaluator Using LLMs in Complex Multi-step Creativity Tasks
The paper introduces CreaEval, an automated creativity evaluator designed for complex multi-step tasks (CGPST). It separates the evaluation process into two phases: a memory‑augmented analysis that transforms responses into structured evidence, and an evidence‑based judging step that scores without seeing raw outputs. Experiments show CreaEval outperforms existing baselines by an average of 22.74% across CGPST and two simpler creativity tasks.
arXiv AI
Aug 26
The Limits of Automatic Evaluation of Creativity in Large Language Models
The paper examines whether existing automatic methods can reliably assess creativity in text produced by large language models (LLMs). By collecting human ratings on 11 creativity dimensions for both human and AI short stories, the authors compare these judgments with automated metrics and LLM-as-a-Judge evaluations. The results show a significant misalignment: automated metrics and LLM judges favor AI-generated stories and show near-zero correlation with human assessments, revealing fundamental limitations in current computational approaches to evaluating creative text.
arXiv AI
Jun 16
LLM Jaggedness Unlocks Scientific Creativity
arXiv:2605. 10574v3 Announce Type: replace Abstract: As artificial intelligence advances, models are not improving uniformly.
arXiv AI
Jun 12
CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges
arXiv:2603. 11863v2 Announce Type: replace Abstract: The saturation of high-quality pre-training data has shifted research focus toward evolutionary systems capable of continuously generating novel artifacts, leading to the success of AlphaEvolve.
arXiv Computation and Language
Oct 12
Evaluating LLM Creativity through Long-Tail Performance
The paper proposes that creativity in large language models can be measured by performance on the long tail of a reference input-output distribution, using n‑gram absence from pretraining data as a novelty proxy. Across 15 benchmarks and 26 open‑weight models, the authors find that creativity benchmarks occupy further tail regions than standard benchmarks and that higher‑tail subsets better align with human creativity judgments. They introduce a new diagnostic, q@Tail(α), to assess mean task quality on the tail fraction of existing benchmarks, revealing significant potential for improving LLM creativity through stronger tail performance.
arXiv AI
Aug 14
LigBench: A Unified and Human-Aligned Benchmark for LLM-based Research Idea Generation
arXiv:2608. 13136v1 Announce Type: cross Abstract: With the rapid advancement of large language models (LLMs), research idea generation has attracted increasing attention.