arXiv:2608.23124v1 Announce Type: cross Abstract: Personalized text generation for authors and literary writing is essential for applications such as adaptive writing assistants, creative support too...
The paper "Evaluating Style-Personalized Text Generation: Challenges and Directions" examines the difficulties of assessing text that is tailored to individual users’ styles. It critiques common metrics such as BLEU, embeddings, and LLM-as-judges, and introduces a style discrimination benchmark covering domain discrimination, authorship attribution, and LLM-generated personalized versus non-personalized discrimination across eight writing tasks. The study finds that ensembles of diverse evaluation metrics outperform single-evaluator approaches and offers guidance for reliable assessment of style-personalized generation.
The paper examines whether existing automatic methods can reliably assess creativity in text produced by large language models (LLMs). By collecting human ratings on 11 creativity dimensions for both human and AI short stories, the authors compare these judgments with automated metrics and LLM-as-a-Judge evaluations. The results show a significant misalignment: automated metrics and LLM judges favor AI-generated stories and show near-zero correlation with human assessments, revealing fundamental limitations in current computational approaches to evaluating creative text.
arXiv:2310.00436v2 Announce Type: replace Abstract: Authorship identification uses patterns in writing to infer who wrote a text, but those patterns also reflect topic, genre, and register. This surv...
arXiv:2605. 17064v2 Announce Type: replace Abstract: Large language models are optimized for instruction following and agentic tasks remain poorly aligned with the requirements of high-quality creative writing.
The paper investigates whether activation steering can capture the complex, multidimensional nature of authorship style. By using structured contrastive prompting along rhetorically motivated dimensions, the authors construct rich style representations directly in activation space, revealing a shared authorship backbone with aspect‑specific residuals. They propose Aspect‑Aware Activation Steering (A3S), a training‑free framework that merges per‑aspect contrastive directions, employs interference‑aware aggregation, and tunes steering strength per instance, achieving better authorship style transfer and outperforming a trained baseline on out‑of‑domain benchmarks.
HyperPersona is a multi‑level hypergraph framework that models the hierarchical structure of text—document, sentence, and word—using hyperedges for documents and sentences and nodes for words. It employs a transformer‑based graph encoder to capture interactions across these linguistic layers, producing context‑sensitive, structurally grounded features for automatic personality prediction. Experiments on the Big Five personality dimensions demonstrate that HyperPersona, relying solely on text, outperforms state‑of‑the‑art baselines by effectively integrating multi‑level linguistic cues.
The paper introduces VOLM, a framework that quantifies how much original value a human adds to a document beyond what a language model could generate from a task description alone. Unlike existing tools that focus on stylistic detection, VOLM extracts content at varying granularities, reconstructs it with an LLM, and compares these reconstructions to those derived from the task description. Evaluations across news articles, ICLR peer reviews, and argumentative essays show that VOLM can distinguish human-authored texts from LLM-generated ones while remaining robust to content-preserving transformations.
arXiv:2608.30754v1 Announce Type: new Abstract: Evaluating creativity in large language model (LLM) outputs remains challenging because creativity is multidimensional and human-centered. We examine h...
Evaluating creativity in large language model (LLM) outputs remains challenging because creativity is multidimensional and human-centered. We examine how reliably LLMs evaluate short literary text in...
arXiv:2608. 19746v1 Announce Type: new Abstract: Personalized text generation aims to make LLMs write in a specific individual's style, yet existing benchmarks measure task accuracy or preference alignment rather than whether the model's output actually resembles the target author's writing.
arXiv:2610.19166v1 Announce Type: new Abstract: Meta-evaluation of text generation metrics and LLM-judges is primarily conducted using metric-human correlations. However, prior work has shown that co...