BLiMPPI is a multilingual benchmark comprising over 12,000 minimal pairs for negative polarity items (NPIs) across 33 languages. It is constructed by extracting NPIs from Universal Dependencies treebanks, labeling their licensing contexts, and generating minimal pair perturbations. The study evaluates 12 multilingual and 2 monolingual language models, revealing that multilingual models perform poorly on low‑resource languages and on pairs that cannot be solved by simple n‑gram patterns, while also exploring how NPI typology, language family, licensing variability, and NPI‑licenser distance affect performance.
By Arzu Burcu G\"uven, Anna Rogers, Leonie Weissweiler, Joakim Nivre, Rob van der Goot
arXiv:2605. 02608v2 Announce Type: replace-cross Abstract: Transformer-based models achieve state-of-the-art dependency parsing for high-resource languages, yet their advantage over simpler architectures in low-resource settings remains poorly understood.
By Kevin Guan, Happy Buzaaba, Christiane Fellbaum
arXiv:2610.08851v1 Announce Type: new
Abstract: Quantifying language distance among closely related languages remains a core challenge in quantitative linguistics. Our previous work [1] introduced Qu...
By Yiping Bai
arXiv:2606. 12708v1 Announce Type: cross Abstract: Despite their linguistic diversity and global significance, African languages remain underrepresented in research and resources to support NLP.
By Happy Buzaaba, Cheikh Mouhamadou Bamba Dione, David Ifeoluwa Adelani, Sylvain Kahane, Kim Gerdes, Bruno Guillaume, Kevin Guan, Aremu Anuoluwapo, Naome A. Etori, Shamsuddeen Hassan Muhammad, Utitofon Inyang, Peter Nabende, David Sabiiti Bamutura, Andiswa Bukula, Chinedu Uchechukwu, Rooweither Mabuya, Idris Akinade, Christiane Fellbaum
arXiv:2610.16837v1 Announce Type: cross
Abstract: Syntactic language models (SLMs) that generate sentences along with their syntactic tree structures have shown promise in both syntactic generalizati...
By Pengcheng Wang, Zhexuan Zhang, Chao Lou, Kewei Tu
The paper introduces MADARTB, the first large-scale parallel Arabic dependency treebank covering five city dialects—Beirut, Cairo, Doha, Rabat, and Tunis—alongside Modern Standard Arabic. It contains 5,000 sentences annotated with the CATiB dependency scheme and shows that while overall construction profiles are similar across varieties, local syntactic structures often diverge, indicating significant structural variation. A supervised parser achieves 87.5% LAS, outperforming the best LLM setup at 85.7% LAS across all six varieties, and the authors release the annotations and models publicly.
By Noor Abo Mokh, Mohamed Adel, Jamila El-Gizuli, Go Inoue, Ahmed Elshabrawy, Nizar Habash
The paper investigates whether language models can transfer hierarchical syntactic knowledge from a source language (L1) to a target language (L2) using a novel cross‑lingual poverty‑of‑stimulus (CPOS) experimental design. Results show that models can partially transfer L1 structural evidence to L2, producing a small but detectable hierarchical bias that varies across language pairs. The study further demonstrates that vocabulary overlap is crucial for successful transfer, indicating that cross‑lingual learning of hierarchical generalization is conditionally achievable.
By Shin Yamaguchi, Masaya Taniguchi, Tatsuki Kuribayashi, Taro Watanabe, Yoshinobu Kano
arXiv:2510.12306v4 Announce Type: replace
Abstract: As natural language corpora expand at an unprecedented rate, manual annotation remains a significant methodological bottleneck in corpus linguistic...
By Cameron Morin, Matti Marttinen Larsson
FreqBLiMP is a frequency‑controlled extension of the BLiMP minimal‑pair benchmark that regenerates all 67 paradigms under explicit Zipf‑frequency regimes while preserving grammatical contrasts. The study evaluates multiple open‑weight LLM families and finds that lower lexical frequency consistently reduces sentence likelihood, yet overall contrastive acceptability accuracy drops only modestly. However, the stability in aggregate accuracy hides significant variability across linguistic phenomena, with models remaining robust on overt morphosyntactic generalization but degrading on lemma‑specific tasks.
By Tyrone White, Yuki Arase
Preference optimisation has proven effective for improving large language models but typically relies on costly human preference annotations. Extending these methods to morphologically rich, low-resource languages remains challenging because such annotations are scarce.
arXiv:2608. 06179v1 Announce Type: new Abstract: Preference optimisation has proven effective for improving large language models but typically relies on costly human preference annotations.
By Hoda Fakharzadehjahromy, Emil Wiman, Andreas Bueff, Hafsteinn Einarsson, Fredrik Heintz
The paper investigates how to fairly compare language models across languages, noting that current evaluation methods vary widely and lack empirical validation. By training controlled monolingual models on parallel data and testing multilingual LLMs, the authors find that many normalized metrics suffer from biases due to tokenization, encoding, and orthographic differences. Instead, they recommend using sentence‑level negative log‑likelihood over semantically equivalent sequences for more reliable cross‑lingual comparisons.
By Xiulin Yang, Ethan Gotlieb Wilcox, Catherine Arnett