The study investigates whether language models trained specifically on Cantonese better predict human reading patterns by comparing eye-tracking data with information-theoretic metrics derived from several models. Two adaptation contrasts were examined: a lightly adapted CKIP GPT-2 Tiny versus its Cantonese derivative JED351, and a heavily adapted Qwen2.5-7B versus CantoneseLLM-7B. Results show that the extensively Cantonese-trained CantoneseLLM-7B consistently outperforms others on lexical surprisal and joint metrics, while entropy reduction favors the less adapted CKIP model, indicating that deeper Cantonese-specific training can improve predictive alignment but that rankings vary by metric.
arXiv:2610.04898v2 Announce Type: replace-cross Abstract: This study analyzes the predictive power of LM-derived, token-level surprisal on Mandarin Chinese reading times. We first propose the Shortes...
arXiv:2610.08879v1 Announce Type: new Abstract: Pretraining strategies significantly impact the quality of language models, yet existing comparisons of Masked Language Modeling (MLM), Whole Word Mask...
arXiv:2609.18156v1 Announce Type: new Abstract: Teochew has a substantial speaker community and exhibits distinctive lexical, syntactic, and pragmatic features, yet textual resources for evaluating l...
Teochew has a substantial speaker community and exhibits distinctive lexical, syntactic, and pragmatic features, yet textual resources for evaluating large language models remain limited. We present T...
The study examines how reader proficiency influences the relationship between layer-wise surprisal from large language models (LLMs) and eye-tracking gaze measures. Using the MECO L2 corpus, researchers compared high- and low-proficiency readers on first-pass gaze duration (FPGD) and total gaze duration (TGD), finding that lower-proficiency readers exhibit deeper Predictive Depth for FPGD, while TGD shows deeper Predictive Depth across both groups. The results suggest that the distribution of predictive power across LLM layers relates to the timing and breadth of reading processes and varies with reader proficiency.
arXiv:2608.30583v1 Announce Type: new Abstract: Standard language proficiency tests rely on linguistic tasks such as vocabulary, grammar and reading comprehension quizzes. An alternative, cognitively...
arXiv:2608. 14896v1 Announce Type: cross Abstract: Large language models work well on English and behave in poorly understood ways on languages typologically far from it.
The study examines whether bilingual language models can reflect the influence of a reader’s first language (L1) on second-language (L2) reading. By comparing bilingual models trained on an L1 that matches the readers’ backgrounds to monolingual English models and bilingual models trained on other L1s, the authors find that matching L1 improves predictions of L2 reading times, especially in late-stage eye-tracking measures and among intermediate-proficiency readers. The effect is modulated by the overlap between L1 and L2 scripts, indicating that bilingual models capture cross-linguistic influence in L2 reading behavior.
arXiv:2611.05803v1 Announce Type: new Abstract: How much phonological knowledge do language models (LMs) possess about Chinese characters, and how is this knowledge represented and used? We introduce...
The study compares four approaches for Chinese sentence-level metaphor identification: BERT fine‑tuning, QLoRA-based large language model fine‑tuning, zero‑shot LLM prompting, and zero‑shot prompting with an expert‑informed procedural Skill. Results show that fine‑tuning yields the highest accuracy on the native test set, while the Skill‑based zero‑shot method provides the most stable performance across three datasets, achieving the highest external floor and the smallest performance range. Adding the Skill reduces false positives on one dataset but increases false negatives on others, indicating a trade‑off between precision and recall.
arXiv:2607. 08152v1 Announce Type: cross Abstract: On the recent EyeBench benchmark, predicting reading comprehension from eye movements exposes a stark gap: text-aware models using pretrained language models reach 56--63% AUROC, while gaze-only models operate at chance.
arXiv:2608.23818v2 Announce Type: replace Abstract: Transformer-based language models are widely used as models of human language processing, yet their attention mechanisms allow lossless access to t...