The study examines how reader proficiency influences the relationship between layer-wise surprisal from large language models (LLMs) and eye-tracking gaze measures. Using the MECO L2 corpus, researchers compared high- and low-proficiency readers on first-pass gaze duration (FPGD) and total gaze duration (TGD), finding that lower-proficiency readers exhibit deeper Predictive Depth for FPGD, while TGD shows deeper Predictive Depth across both groups. The results suggest that the distribution of predictive power across LLM layers relates to the timing and breadth of reading processes and varies with reader proficiency.
By Akio Hayakawa, Horacio Saggion
The study investigates whether language models trained specifically on Cantonese better predict human reading patterns by comparing eye-tracking data with information-theoretic metrics derived from several models. Two adaptation contrasts were examined: a lightly adapted CKIP GPT-2 Tiny versus its Cantonese derivative JED351, and a heavily adapted Qwen2.5-7B versus CantoneseLLM-7B. Results show that the extensively Cantonese-trained CantoneseLLM-7B consistently outperforms others on lexical surprisal and joint metrics, while entropy reduction favors the less adapted CKIP model, indicating that deeper Cantonese-specific training can improve predictive alignment but that rankings vary by metric.
By Ziqi Zhang, Emmanuele Chersoni, Mohammad Momenian
arXiv:2505.12196v2 Announce Type: replace
Abstract: The impressive linguistic abilities of large language models (LLMs) have recommended them as models of human sentence processing, with some conject...
By Yi-Chien Lin, Hongao Zhu, William Schuler
arXiv:2607. 08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance.
By Sumin Lee, Kyeonghun Kim, Subeen Lee, Jiwon Yang, Tien Nguyen, Ken Ying-Kai Liao, Nam-Joon Kim
arXiv:2610.08879v1 Announce Type: new
Abstract: Pretraining strategies significantly impact the quality of language models, yet existing comparisons of Masked Language Modeling (MLM), Whole Word Mask...
By Yiping Bai
arXiv:2607. 03994v1 Announce Type: cross Abstract: Modern language models generally represent text as sequences of discrete token embeddings, an assumption deeply rooted in current practice but rarely questioned.
By Shuyang Xiang, Hao Guan
arXiv:2609. 21362v1 Announce Type: new Abstract: Conventional tokenizers represent text as characters or statistically derived subwords, overlooking the internal phonological structure of syllables and often requiring large vocabularies.
By Nghia Hieu Nguyen, Thai Bao Huynh, Binh-An Dinh-Le, Phu Gia Hoang, Dat Tien Nguyen, Kiet Van Nguyen, Ngan Luu-Thuy Nguyen
The study introduces a latent‑process multinomial processing tree (MPT) model to analyze human reading and comprehension of garden‑path sentences across four reading paradigms (eye tracking, uni‑ and bidirectional self‑paced reading, Maze). The model separates the likelihood of an incorrect initial analysis, the cost of encountering an incompatible continuation, and the cost of syntactic reanalysis, yielding more realistic parameter estimates when inattentive trials are considered. Cross‑validation shows that this MPT model predicts human reading patterns and end‑of‑trial judgments better than a model relying solely on large‑language‑model (LLM) surprisal, and that incorporating surprisal as an additional predictor further improves fit.
By Dario Paape, Tal Linzen, Shravan Vasishth
Vision‑Language Models (VLMs) are increasingly replacing traditional OCR for document understanding, but this study shows they often rewrite imperfect text into more plausible forms, a flaw that clean‑text OCR benchmarks miss. The authors created FaithC4, a multilingual perturbation benchmark of 1,455 single‑page documents with scramble, random substitution, and visually similar substitution attacks, and evaluated 15 systems across general‑purpose VLMs, OCR‑specialized VLMs, and traditional OCR pipelines. Results reveal that general‑purpose VLMs suffer up to 6.9 WER points under perturbation, OCR‑specialized VLMs 0.1–3.4 points, and traditional OCR less than 0.8 points on English; probing Qwen3‑VL‑4B shows rewriting occurs only when a perturbed word’s final‑layer representation remains close to the original, with short words (4–6 characters) rewritten up to 10% of the time.
whyItMatters":"The findings highlight a critical limitation of VLMs in document transcription, underscoring the need for robust evaluation benchmarks that capture rewriting behavior beyond clean‑text accuracy."
By Gwang Gook Lee, Kenan Emir Ak, Jay Mohta, Yan Xu, Dimitrios Dimitriadis
arXiv:2608.30583v1 Announce Type: new
Abstract: Standard language proficiency tests rely on linguistic tasks such as vocabulary, grammar and reading comprehension quizzes. An alternative, cognitively...
By Shachar Frenkel, Ido Falah, Omer Shubi, Yevgeni Berzak
arXiv:2603. 18482v2 Announce Type: replace-cross Abstract: Standard decoding strategies for text generation, including top-$k$, nucleus sampling, and contrastive search, select tokens based on likelihood, restricting outputs to high-probability regions.
By Esteban Garces Arias, Nurzhan Sapargali, Christian Heumann, Matthias A{\ss}enmacher
arXiv:2608. 04021v1 Announce Type: cross Abstract: Cloze-style probes that vary how often a target token appears implicitly assume that more copies of a target affect prediction the same way regardless of where the readout slot sits.
By Han-yu Wang