arXiv Computation and Language
Evaluating Style-Personalized Text Generation: Challenges and Directions
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Aug 25
LITERARYBIGFIVE: Author-Personalized Text Generation in a Unified Interpretable Space
arXiv:2608.23124v1 Announce Type: cross Abstract: Personalized text generation for authors and literary writing is essential for applications such as adaptive writing assistants, creative support too...
Hugging Face Trending Papers
Aug 24
LITERARYBIGFIVE: Author-Personalized Text Generation in a Unified Interpretable Space
Personalized text generation for authors and literary writing is essential for applications such as adaptive writing assistants, creative support tools, and computational literary analysis. However, e...
arXiv Computation and Language
Aug 21
PersonalBench: Measuring the Authorship Gap in LLM Personalization
arXiv:2608. 19746v1 Announce Type: new Abstract: Personalized text generation aims to make LLMs write in a specific individual's style, yet existing benchmarks measure task accuracy or preference alignment rather than whether the model's output actually resembles the target author's writing.
arXiv AI
Aug 26
The Limits of Automatic Evaluation of Creativity in Large Language Models
The paper examines whether existing automatic methods can reliably assess creativity in text produced by large language models (LLMs). By collecting human ratings on 11 creativity dimensions for both human and AI short stories, the authors compare these judgments with automated metrics and LLM-as-a-Judge evaluations. The results show a significant misalignment: automated metrics and LLM judges favor AI-generated stories and show near-zero correlation with human assessments, revealing fundamental limitations in current computational approaches to evaluating creative text.
arXiv AI
2d ago
Value Over Language Model: Detecting Original Contribution in Writing
The paper introduces VOLM, a framework that quantifies how much original value a human adds to a document beyond what a language model could generate from a task description alone. Unlike existing tools that focus on stylistic detection, VOLM extracts content at varying granularities, reconstructs it with an LLM, and compares these reconstructions to those derived from the task description. Evaluations across news articles, ICLR peer reviews, and argumentative essays show that VOLM can distinguish human-authored texts from LLM-generated ones while remaining robust to content-preserving transformations.
arXiv AI
Aug 20
Self- and Other-Labels Induce Bidirectional Bias in LLM Judges
The study investigates bias in large language model (LLM) judges by having ten LLMs evaluate narrative constraint selections rather than generated text. Results show that self-preference largely disappears under blind evaluation when quality and evaluator severity are controlled, but self- and other-labels alone shift scores bidirectionally when quality is matched. The authors conclude that authorship attribution drives evaluation bias and that open-ended, ground‑truth‑free tasks can effectively study LLM judge behavior.