Pearl-Chat is an 8.5‑million‑parameter causal decoder‑only transformer built from first principles in JAX and Flax NNX, specifically engineered for the low‑resource Bantu language Luganda. It uses a byte‑level BPE tokenizer trained from scratch on curated Luganda corpora, achieving a two‑fold compression improvement over larger models like LLaMA‑3 and Gemma. The model is trained in two stages—pre‑training on diverse Luganda text and instruction fine‑tuning on dialogues—reaching a validation perplexity of 1.20 and high ROUGE‑L and BLEU scores on a held‑out diagnostic benchmark, with all weights, tokenizers, and code released openly.
By Wesley Kambale, Roland Sankara
arXiv:2608. 07629v1 Announce Type: cross Abstract: Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon.
By Samiratu Ntohsi, Neza David Tuyishimire, Anesu Kafesu, Marvin Ogore, Samuel Oluwajunwonlo Babalola, Oche Ankeli
We present DONDO, a family of open, permissively licensed automatic speech recognition (ASR) base models for African languages, built on the w2v-BERT 2. 0 self-supervised speech encoder.
arXiv:2606. 24460v1 Announce Type: cross Abstract: Commercial large language models bill, scale latency, and budget context per token.
By Olaoye Anthony Somide
arXiv:2606. 20089v1 Announce Type: cross Abstract: Persian pretrained language models (PLMs) are still limited by the scarcity of large-scale, high-quality pretraining corpora and by insufficient evaluation beyond standard classification and NER tasks.
By Arash Ghafouri, Mahdi Firouzmandi, Hossein Saberi, Mohammad Reza Hasani Ahangar
MoVoC introduces a morphology-aware subword construction method for Ge'ez script languages, integrating supervised morphological analysis with Byte Pair Encoding to preserve morphological boundaries while retaining lexical meaning. The resulting MoVoC-Tok tokenizer outperforms baselines in MorphScore and morpheme boundary precision across several low-resource, morphologically complex languages, notably achieving significant gains on Tigre through cross-lingual application. The authors also release morpheme-annotated datasets and a morpheme-aware vocabulary for public use.
By Hailay Kidu Teklehaymanot, Dren Fazlija, Wolfgang Nejdl
The paper introduces TokenAdapt, a model‑agnostic tokenizer transplantation method that uses a hybrid heuristic to initialize new token embeddings, and a novel pre‑tokenization learning approach for multi‑word Supertokens to improve compression. TokenAdapt combines local subword decomposition and global semantic similarity to preserve semantics while reducing retraining needs. Empirical results show that TokenAdapt outperforms existing baselines such as Transtokenizer and ReTok, achieving lower perplexity ratios and significant compression gains.
By Shaurya Sharthak, Vinayak Pahalwan, Adithya Kamath, Adarsh Shirawalmath
arXiv:2608. 03494v1 Announce Type: cross Abstract: Vocabulary extension is an efficient way to adapt pretrained large language models (LLMs) to new languages, but the initialization of newly added token embeddings can strongly affect continued pre-training (CPT) efficiency.
By Raviraj Joshi, Utkarsh Vaidya, Sanjay Singh Chauhan, Niranjan Wartikar
arXiv:2604. 03532v2 Announce Type: replace-cross Abstract: Large language models (LLMs) show strong multilingual capabilities, yet reliably controlling the language of their outputs remains difficult.
By Sing Hieng Wong, Hassan Sajjad, A. B. Siddique
The study investigates how the choice of tokenizer influences multilingual language models across 54 tokenizers and 123 models. It finds that tokenizer impact is greater for languages with less training data, and that excluding a language from tokenizer training consistently worsens its performance. While reallocating tokenizer training data to lower-resource languages can help, it does not guarantee improvement, and the best tokenizer properties vary by language, enabling predictive screening of tokenizer candidates.
By Clara Meister, G\"ul Sena Alt{\i}nta\c{s}, Antoine Bosselut
TabuLM is a new language model pre‑trained on Kinyarwanda tabular data, extending KinyaBERT‑large with row, column, and cell‑type embeddings and a table‑structure attention bias. It introduces two pre‑training objectives—Masked Cell Recovery and Column Type Prediction—and is trained on 172 Rwandan government tables. On the TabQA‑kin benchmark, TabuLM achieves 62.0% exact match, outperforming KinyaBERT‑large and multilingual baselines by significant margins.
By Ireddi Rakshitha, Devavarapu Yashwanth, Ntakirutimana Pierre
arXiv:2606. 18717v1 Announce Type: cross Abstract: Turkish is agglutinative: meaning is carried by morphemes, yet the subword tokenizers that drive modern language models split words by corpus statistics, fragmenting semantically loaded suffixes and -- in the case of WordPiece and rule-based analyzers -- failing to decode their output back to the original text.
By Tolga \c{S}akar