NaijaFacts is a multilingual benchmark that tests language models on factual and statistical questions about Nigeria, presented in English, Nigerian Pidgin, Yoruba, Igbo, and Hausa. The study shows that models perform much better on UK and US facts (49–50% accuracy) than on Nigerian facts (20% accuracy), indicating a strong knowledge gap. Language effects are smaller, with performance dropping from English to Yoruba, but overall scale does not improve accuracy on Nigerian statistics.
By Zainab Afolabi, Nikita Kozodoi, Jack Butler, Babs Khalidson
Multilingual benchmarks for African languages mostly translate generic questions into local languages, measuring linguistic competence but not whether models know facts about Africa. We introduce \nai...
arXiv:2610.14476v1 Announce Type: cross
Abstract: Comparing large language models can reveal shared representational structure and systematic differences between them. However, aggregate similarity m...
By Jialin Wu, Meenakshi Khosla
arXiv:2606. 03219v1 Announce Type: cross Abstract: African languages have very little labelled data, and it is unclear if augmenting the quantity of annotation data reliably enhances downstream performance.
By Anuj Tiwari, Oluwapelumi Ogunremu, Terry Oko-odion, Jesujuwon Egbewale, Hannah Nwokocha
The paper evaluates four metrics—CKA, ANC, GMM dominance per token, and ILO—used to measure cross‑lingual representation sharing in multilingual language models. Across 21 models ranging from 125 M to 14 B parameters, the metrics disagree, and the authors attribute this to anisotropy, where representations cluster in a narrow embedding cone. Only ILO shows a strong, robust correlation with cross‑lingual transfer performance (Spearman’s ρ = 0.90) after controlling for model size, family, and task variation, leading the authors to recommend ILO as the primary metric alongside anisotropy diagnostics.
By Oskar Holmstr\"om, Marcel Bollmann, Marco Kuhlmann
arXiv:2606. 14347v1 Announce Type: new Abstract: Large language models exhibit strong multilingual capabilities, however, their internal representations are difficult to interpret.
By Boris Marinov, Angira Sharma, Christian Schroeder de Witt, Philip Torr, Anisoara Calinescu, Jialin Yu
The paper investigates how large language models (LLMs) share a common Fisher‑Rao geometry in their next‑token probability distributions, revealing that behaviour largely determines this geometry while activation geometry depends on coordinate choices. Across transformer, state‑space, and recurrent architectures, output geometries align more closely than activation geometries, and this shared structure facilitates semantic‑category transfer and improves agreement with human word choices as models scale and train. The study further demonstrates that geometry can guide minimum‑disturbance interventions, enabling reusable control that preserves behaviour better than Euclidean methods and enhances steering, editing, attribution, dictionary learning, and fine‑tuning.
By Dario Picozzi
The paper investigates covert dialect bias in large language models (LLMs) by analyzing how internal probability distributions associate different English varieties—Standard American English, African American Vernacular English, Nigerian Standard English, and Nigerian Pidgin—with housing-related adjectives. Using 260 meaning‑matched sentence quadruples and log‑probability scoring across ten open‑weight LLMs, the study finds that AAVE and NP are consistently linked to more negative adjectives than SAE, with NP experiencing the greatest penalty. The bias varies by context and stereotype cluster, and Nigerian Standard English shows a context‑dependent shift, being favored in formal tenant screening but penalized in more socially proximate scenarios.
By Chowdhury Mohammad Abdullah, Rita Orji
The paper surveys NLP research on Nigeria’s three major low‑resource languages—Hausa, Yoruba, and Igbo—covering over 500 languages spoken by 175 million people. It reviews 293 studies, finding that only 27.6% produced new linguistic resources, indicating a heavy reliance on repurposing existing data. The authors highlight under‑explored challenges such as morphological analysis and diacritic representation, and call for collaborative resource enrichment and community support to advance NaijaNLP and low‑resource NLP more broadly.
By Isa Inuwa-Dutse
The study investigates whether Modern Standard Arabic (MSA) dominates internal representations in large language models (LLMs) when generating Arabic, even if prompted with dialectal forms. By extending a language-dominance framework to 26 Arabic varieties, the authors find no evidence that MSA serves as a dominant internal representation; instead, dialect representations are densely overlapping, with mutual information dropping sharply compared to more distinct languages. The strongest separability effects vary across layers and architectures, indicating that generation bias toward MSA does not necessarily reflect internal representational dominance.
By Abdu Sallouh, Nicholas Popovi\v{c}, Michael F\"{a}rber
arXiv:2607. 04814v1 Announce Type: cross Abstract: Extending automatic speech recognition (ASR) to low-resource African languages is constrained by the prohibitive demands of data collection at scale.
By Andrei Florian, Cynthia Jayne Amol, Hope Kerubo Ombaba, Xiaoyu Cui, Boniface Mwau, Biatus Maina Kamau, Lilian Diana Awuor Wanzare, Christiane Fellbaum, Happy Buzaaba
arXiv:2607. 15847v1 Announce Type: cross Abstract: Metaphor in Arabic is a culturally grounded mechanism for constructing meaning, encoding cultural knowledge that shapes interpretation.
By Suzan Awinat, Alfonso Ortega del Puente