The study examined how the choice of corpus affects estimates of dependency distance in language. By comparing 38 pairs of treebanks from the same language, the authors found that cross-treebank agreement was only moderate, with nearly 40% of language orderings reversed when switching treebanks. Treebank selection explained about 29% of the variance, a discrepancy that far exceeds within-treebank sampling error and persists across multiple preprocessing settings, yet all treebanks still supported the principle of dependency-length minimization.
By Sirui Chen
arXiv:2606. 12708v1 Announce Type: cross Abstract: Despite their linguistic diversity and global significance, African languages remain underrepresented in research and resources to support NLP.
By Happy Buzaaba, Cheikh Mouhamadou Bamba Dione, David Ifeoluwa Adelani, Sylvain Kahane, Kim Gerdes, Bruno Guillaume, Kevin Guan, Aremu Anuoluwapo, Naome A. Etori, Shamsuddeen Hassan Muhammad, Utitofon Inyang, Peter Nabende, David Sabiiti Bamutura, Andiswa Bukula, Chinedu Uchechukwu, Rooweither Mabuya, Idris Akinade, Christiane Fellbaum
The paper presents a pipeline that leverages large language models to extract grammatical rules, example sentences, and lexicons from descriptive grammar books, producing synthetic parallel corpora for fine‑tuning machine translation models. Evaluated on three low‑resource languages—Kalamang, Tuatschin, and Mandan—the synthetic data improves translation quality over seed‑data baselines in 75% of configurations for Kalamang and 59% for Tuatschin, achieving up to +8.8 ChrF++ gains. A factorial study across 96 configurations identifies which combinations of target part‑of‑speech, retrieval granularity, and sample volume drive performance gains and where they fail, demonstrating that static linguistic documentation can be repurposed for practical translation tools for severely under‑resourced languages.
By Varun Ghat Ravikumar, Sina Ahmadi, Lena J\"ager, Rico Sennrich
The paper investigates how to fairly compare language models across languages, noting that current evaluation methods vary widely and lack empirical validation. By training controlled monolingual models on parallel data and testing multilingual LLMs, the authors find that many normalized metrics suffer from biases due to tokenization, encoding, and orthographic differences. Instead, they recommend using sentence‑level negative log‑likelihood over semantically equivalent sequences for more reliable cross‑lingual comparisons.
By Xiulin Yang, Ethan Gotlieb Wilcox, Catherine Arnett
arXiv:2605. 02608v2 Announce Type: replace-cross Abstract: Transformer-based models achieve state-of-the-art dependency parsing for high-resource languages, yet their advantage over simpler architectures in low-resource settings remains poorly understood.
By Kevin Guan, Happy Buzaaba, Christiane Fellbaum
NEPSEE is a Nepali sentiment analysis benchmark created by having two annotators independently label 5,614 tweets from Nepali public figures on a five‑point scale. Only 3,189 sentences with identical polarity are kept for a three‑class version, while 766 sentences with adjacent labels are used in a five‑point task. Baseline results show NepBERTa achieving a macro‑F1 of 0.877 on the three‑class benchmark, with commercial LLMs performing similarly; on the five‑point task, Claude Opus 5.5 outperforms encoders by 0.10–0.12 macro‑F1, and a tokenizer audit reveals that restoring Devanagari vowel signs can significantly alter performance.
By Gaurab Baral, Mimansak Nepal, Grishma Basnet
arXiv:2607. 23319v1 Announce Type: cross Abstract: Standard subword tokenization algorithms such as Byte-Pair Encoding (BPE) and SentencePiece are trained predominantly on modern language corpora and produce inefficient segmentations when applied to classical Indian languages.
By Poornima Kumaresan, Pavithra Muruganantham, Lakshmi Rajendran, Santhosh Sivasubramani
arXiv:2608. 07629v1 Announce Type: cross Abstract: Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon.
By Samiratu Ntohsi, Neza David Tuyishimire, Anesu Kafesu, Marvin Ogore, Samuel Oluwajunwonlo Babalola, Oche Ankeli
ThaiTrees is a 342‑million‑token corpus of Thai text spanning news, Wikipedia, spoken transcripts, and social media, automatically parsed under the Universal Dependencies framework. The authors provide a reproducible pipeline for cleaning, processing, and parsing the data, and release the resulting CoNLL‑U files and a frequency lexicon in machine‑readable formats. This resource enables researchers to search grammatical relations and study syntactic distributions at scale.
By Attapol T. Rutherford, Papatchol Thientong
FreqBLiMP is a frequency‑controlled extension of the BLiMP minimal‑pair benchmark that regenerates all 67 paradigms under explicit Zipf‑frequency regimes while preserving grammatical contrasts. The study evaluates multiple open‑weight LLM families and finds that lower lexical frequency consistently reduces sentence likelihood, yet overall contrastive acceptability accuracy drops only modestly. However, the stability in aggregate accuracy hides significant variability across linguistic phenomena, with models remaining robust on overt morphosyntactic generalization but degrading on lemma‑specific tasks.
By Tyrone White, Yuki Arase
The paper audits 51 recent multilingual evaluation benchmarks covering 242 datasets and 219 languages, focusing on coverage, representativeness, and rigor. It finds that while coverage is broad, many languages appear in only one benchmark and entire regions are underrepresented; low‑resource languages face a task equity gap. Representativeness is compromised by heavy reliance on English translations, and rigor is weakened by benchmark contamination and parallel‑corpus overlap. The authors propose concrete recommendations for building evaluation frameworks that are natively constructed, culturally grounded, contamination‑aware, and community‑aligned.
By Manan Uppadhyay, Himanshu Beniwal, Prashant Kodali, Sunayana Sitaram
Multilingual evaluation benchmarks are the primary instrument for assessing whether large language models generalize beyond English, yet the adequacy of these benchmarks has received little systematic...