← Back to all news
arXiv Computation and Language

The Cognitive Cost of LLM Writing: A Reading-Time Comparison with Human Text

The study examines how reading times differ between human-written texts and rewrites generated by large language models (LLMs). Using a RoBERTa-RT model trained on eye-tracking data, the authors compare predicted reading times for 496 paired articles and find that LLM rewrites are shorter overall but have higher predicted reading times per word. These differences correlate with linguistic changes such as longer, less frequent words, greater lexical diversity, and higher contextual predictability.

Related stories

arXiv AI
Oct 7

STRUCTURALCOST: A controlled reading time dataset for modeling human sentence processing difficulty

STRUCTURALCOST is a self‑paced reading dataset comprising 475 participants and 40,800 observations that isolates the processing cost of long‑distance subject‑verb dependency resolution. The study replicates a known psycholinguistic finding at NLP scale: human reading times at the main verb increase with dependency length, driven by syntactic embedding beyond linear distance. Various language models—n‑gram, SSMs, and transformers—partially mirror this graded difficulty profile but consistently underestimate the integration cost humans incur, a gap that persists across architectures and model sizes.

By Nina Nusbaumer, Iria de-Dios-Flores, Corentin Bel, Christophe Pallier, Guillaume Wisniewski, Beno\^it Crabb\'e
llmsragnlp
More like this →
arXiv Computation and Language
Sep 25

LLM surprisal is necessary but not sufficient to capture English garden-path effects: Evidence from joint latent modeling of reading paradigms

The study introduces a latent‑process multinomial processing tree (MPT) model to analyze human reading and comprehension of garden‑path sentences across four reading paradigms (eye tracking, uni‑ and bidirectional self‑paced reading, Maze). The model separates the likelihood of an incorrect initial analysis, the cost of encountering an incompatible continuation, and the cost of syntactic reanalysis, yielding more realistic parameter estimates when inattentive trials are considered. Cross‑validation shows that this MPT model predicts human reading patterns and end‑of‑trial judgments better than a model relying solely on large‑language‑model (LLM) surprisal, and that incorporating surprisal as an additional predictor further improves fit.

By Dario Paape, Tal Linzen, Shravan Vasishth
llms
More like this →
arXiv Computation and Language
Sep 30

Reader Proficiency Shapes Layer-wise Surprisal Profiles

The study examines how reader proficiency influences the relationship between layer-wise surprisal from large language models (LLMs) and eye-tracking gaze measures. Using the MECO L2 corpus, researchers compared high- and low-proficiency readers on first-pass gaze duration (FPGD) and total gaze duration (TGD), finding that lower-proficiency readers exhibit deeper Predictive Depth for FPGD, while TGD shows deeper Predictive Depth across both groups. The results suggest that the distribution of predictive power across LLM layers relates to the timing and breadth of reading processes and varies with reader proficiency.

By Akio Hayakawa, Horacio Saggion
llms
More like this →
arXiv AI
Jul 10

LEXIC: Lightweight Eye-tracking eXtension via Injected Complexity

arXiv:2607. 08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance.

By Sumin Lee, Kyeonghun Kim, Subeen Lee, Jiwon Yang, Tien Nguyen, Ken Ying-Kai Liao, Nam-Joon Kim
llmsbenchmarks
More like this →
arXiv Computation and Language
Aug 27

Distinct dynamics of conceptual and referential disruptions in human reading and large language model processing

The study investigates how disrupting conceptual versus referential information in short narratives affects human reading and large language model (LLM) processing. In humans, conceptual disruptions cause a strong, localized processing cost that peaks early and declines quickly, while referential disruptions produce weaker, gradually decreasing effects that are more influenced by sentence boundaries. In LLMs, both disruptions appear immediately at the manipulated word; surprisal patterns mirror human reading, whereas output-layer representations show that referential disruption initially causes a larger displacement before both types decay following a power-law.

By Rui He, Nihal Altay, Wolfram Hinzen
llms
More like this →
arXiv Computation and Language
5d ago

Why is AI-generated language so hard to understand?

The paper investigates why AI-generated text has become harder to understand, focusing on psycholinguistic features linked to human processing difficulty such as object relative clauses, long syntactic dependencies, and low-frequency n-grams. Using a new corpus of AI-generated text from diverse domains, the study finds that frontier and near-frontier language models employ these difficult features more frequently than paraphrases from round-trip translation and human writing, including more inanimate subjects with animate verbs, longer dependencies, stacked nouns, and reduced object relatives. The authors hypothesize that AI models prioritize information density for tasks like math, code, and reasoning, diverging from human communication goals, and demonstrate this by fine‑tuning a model with a length penalty, which reduces human‑like linguistic behavior.

By Thomas Hikaru Clark, Matias B. Martin, Kyle Mahowald
llmsfine-tuning
More like this →