The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
The paper presents a detailed examination of narrative elements—agency, setting, and events—within the Dolma web-scale pretraining corpus. Using a framework of 11 interpretable dimensions, the authors hand‑annotated 400 passages, expanded this to a 25,000‑passage LLM‑labeled dataset, and trained NarraBERT models to predict narrative features across 13 million passages, producing the NarraDolma dataset. The study reveals that narrative structure is measurable at scale and that narrative qualities vary unevenly across different data sources, topics, and formats, highlighting gaps in current data curation practices.
arXiv:2609.38406v1 Announce Type: new Abstract: Access to real-world information is often noisy and fragmented. Constructing a coherent narrative from such fragments requires models to reconstruct mi...
arXiv:2602. 15851v2 Announce Type: replace-cross Abstract: Applications of narrative theories using large language models (LLMs) deliver promising methods in automatic story generation and understanding tasks.
The paper explores whether language models encode components that represent narrativity. By examining GPT‑2‑Small, Gemma‑2‑2B, and Llama‑3.1‑8B, the authors identify specific attention heads, MLP neurons, and sparse autoencoder features whose activations correlate with narrative properties. Ablating these components raises next‑token prediction loss more for high‑narrativity texts and diminishes narrativity in open‑ended generations, with the strongest effects seen in story completion and narrative question answering tasks.
arXiv:2608. 15654v1 Announce Type: cross Abstract: Large language models can write fluent stories, but open-ended storytelling requires more than local fluency.
arXiv:2601. 17226v2 Announce Type: replace-cross Abstract: Counterfactual story retelling exposes LLM shortcomings in constrained narrative solution spaces where they can no longer rely on recalling memorised training data.