arXiv AI

Towards Human-Level Book-Writing Capability

arXiv:2605. 17064v2 Announce Type: replace Abstract: Large language models are optimized for instruction following and agentic tasks remain poorly aligned with the requirements of high-quality creative writing.

arXiv AI
Aug 19

Attention Flows: Tracing LLM Conceptual Engagement via Story Summaries

The paper investigates how large language models (LLMs) engage with long-form narratives by comparing their generated novel summaries to human-authored ones. Researchers align sentences from 150 human-written summaries to specific chapters, highlighting the challenge of this alignment task and the complexity of summarization. They find stylistic differences and that LLMs tend to focus more on the ends of texts, suggesting insights into why models may struggle with narrative comprehension.

By Rebecca M. M. Hicke, Sil Hamilton, David Mimno, Ross Deans Kristensen-McLachlan
arXiv AI
Jul 22

PlotTwist: A Creative Plot Generation Framework with Small Language Models

arXiv:2603. 16410v2 Announce Type: replace-cross Abstract: Creative plot generation presents a fundamental challenge for language models: transforming a concise premise into a coherent narrative that sustains global coherence, character development, pacing, tone consistency, and emotional progression.

By Abhinav Thorat, Ravi Kolla, Jyotin Goel, Madhav Kataria, Niranjan Pedanekar
arXiv Computation and Language
Sep 11

Characterizing Narrative Content in Web-scale LLM Pretraining Data

The paper presents a detailed examination of narrative elements—agency, setting, and events—within the Dolma web-scale pretraining corpus. Using a framework of 11 interpretable dimensions, the authors hand‑annotated 400 passages, expanded this to a 25,000‑passage LLM‑labeled dataset, and trained NarraBERT models to predict narrative features across 13 million passages, producing the NarraDolma dataset. The study reveals that narrative structure is measurable at scale and that narrative qualities vary unevenly across different data sources, topics, and formats, highlighting gaps in current data curation practices.

By Teagan Johnson, Elliott Ash, Andrew Piper, Maria Antoniak
arXiv AI
Sep 2

Value Over Language Model: Detecting Original Contribution in Writing

The paper introduces VOLM, a framework that quantifies how much original value a human adds to a document beyond what a language model could generate from a task description alone. Unlike existing tools that focus on stylistic detection, VOLM extracts content at varying granularities, reconstructs it with an LLM, and compares these reconstructions to those derived from the task description. Evaluations across news articles, ICLR peer reviews, and argumentative essays show that VOLM can distinguish human-authored texts from LLM-generated ones while remaining robust to content-preserving transformations.

By Vibhhu Sharma, Thorsten Joachims, Sarah Dean
arXiv AI
Jun 4

POLARIS: Guiding Small Models to Write Long Stories

arXiv:2606. 04095v1 Announce Type: cross Abstract: Small open-weight models struggle at long-form creative writing: their generated stories either fall far short of the requested length, or their quality significantly degrades as length increases, especially when compared to frontier models.

By Rishanth Rajendhran, Jenna Russell, Mohit Iyyer, John Frederick Wieting