arXiv:2608.23818v2 Announce Type: replace
Abstract: Transformer-based language models are widely used as models of human language processing, yet their attention mechanisms allow lossless access to t...
By Lanni Bu, Xiulin Yang, Christian Clark, Alex Warstadt, Ethan Gotlieb Wilcox
The study evaluates eleven autoregressive transformer models on English agreement attraction scenarios using a surprisal-based approach. Results show that while transformers match human reading times for prepositional phrase configurations, they perform poorly on object‑extracted relative clauses, with predictions diverging across models and failing to capture human interference patterns. The authors argue that current transformers cannot adequately model human morphosyntactic processing and call for more rigorous, comprehensive testing to avoid misleading conclusions from limited syntactic setups.
By Titus von der Malsburg, Sebastian Pad\'o
STRUCTURALCOST is a self‑paced reading dataset comprising 475 participants and 40,800 observations that isolates the processing cost of long‑distance subject‑verb dependency resolution. The study replicates a known psycholinguistic finding at NLP scale: human reading times at the main verb increase with dependency length, driven by syntactic embedding beyond linear distance. Various language models—n‑gram, SSMs, and transformers—partially mirror this graded difficulty profile but consistently underestimate the integration cost humans incur, a gap that persists across architectures and model sizes.
By Nina Nusbaumer, Iria de-Dios-Flores, Corentin Bel, Christophe Pallier, Guillaume Wisniewski, Beno\^it Crabb\'e
The study examines whether Dutch language models exhibit coherence‑illusion effects similar to human readers, using texts that refer back to earlier context with words like ‘again’ and ‘too’. Surprisal at the critical word aligns with human acceptability and eye‑tracking data, showing that models are more surprised by incoherent continuations unless a matching distractor is present. Attention entropy and an energy metric from associative‑memory literature reveal heads that behave differently under coherence versus incoherence, and ablating these heads demonstrates transfer effects across experiments, indicating a shared underlying mechanism.
By Ece Takmaz, Nitin Kumar, Li Kloostra, Jakub Dotlacil
arXiv:2510. 05969v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed on complex reasoning tasks, yet little is known about their ability to internally evaluate problem difficulty, which is an essential capability for adaptive reasoning and efficient resource allocation.
By Sunbowen Lee, Qingyu Yin, Chak Tou Leong, Jialiang Zhang, Yicheng Gong, Shiwen Ni, Min Yang, Xiaoyu Shen
arXiv:2505.12196v2 Announce Type: replace
Abstract: The impressive linguistic abilities of large language models (LLMs) have recommended them as models of human sentence processing, with some conject...
By Yi-Chien Lin, Hongao Zhu, William Schuler