KinyaEmbed is the first sentence‑embedding model specifically designed for Kinyarwanda, built on KinyaBERT‑large and trained through a four‑stage curriculum that incorporates paraphrase pairs, translated MNLI triplets, OPUS‑100 translation pairs, and high‑quality KinyaCOMET pairs. It outperforms existing multilingual embeddings on the SemRel2024‑rw benchmark, achieving a Spearman ψ of 0.7298, and introduces the Wiki‑RW‑STS benchmark of 300 contamination‑free Kinyarwanda sentence pairs. All model checkpoints, filtered pairs, and the new benchmark are publicly released.
We present TabuLM, the first language model pre-trained on Kinyarwanda tabular data. Kinyarwanda is a morphologically rich Bantu language spoken by over 12 million people in Rwanda, yet lacks any dedi...
TabuLM is a new language model pre‑trained on Kinyarwanda tabular data, extending KinyaBERT‑large with row, column, and cell‑type embeddings and a table‑structure attention bias. It introduces two pre‑training objectives—Masked Cell Recovery and Column Type Prediction—and is trained on 172 Rwandan government tables. On the TabQA‑kin benchmark, TabuLM achieves 62.0% exact match, outperforming KinyaBERT‑large and multilingual baselines by significant margins.
arXiv:2608. 07629v1 Announce Type: cross Abstract: Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon.
The paper compares generative and encoder-based neural models for multilingual Named Entity Recognition (NER) across the eleven languages of the Naamapadam benchmark. Five classic model families, four decoder-only large language models fine‑tuned with LoRA and 4‑bit NF4 quantisation, and nine generative models in zero‑to‑5‑shot inference were evaluated under strict CoNLL span‑level metrics. Encoder-based models (mBERT and XLM‑R) achieved substantially higher F1 scores—up to 0.675 on Hindi—than any generative architecture, with gaps of 7.5–40 percentage points; the best few‑shot result reached only 28% of the encoder baseline. The study identifies three language clusters (encoder‑dominant, partial‑coverage, and failure‑zone) and offers deployment guidelines based on transfer learning and low‑resource NLP principles.
arXiv:2606. 25365v2 Announce Type: replace-cross Abstract: We present a study on low-resource machine translation for the Tangkhul-English (nmf-en) language pair.
arXiv:2608.12018v2 Announce Type: replace Abstract: Neural Machine Translation (NMT) and Large Language Models (LLMs) excel at cross-lingual tasks but often fail to capture intra-lingual morphologica...
Pearl-Chat is an 8.5‑million‑parameter causal decoder‑only transformer built from first principles in JAX and Flax NNX, specifically engineered for the low‑resource Bantu language Luganda. It uses a byte‑level BPE tokenizer trained from scratch on curated Luganda corpora, achieving a two‑fold compression improvement over larger models like LLaMA‑3 and Gemma. The model is trained in two stages—pre‑training on diverse Luganda text and instruction fine‑tuning on dialogues—reaching a validation perplexity of 1.20 and high ROUGE‑L and BLEU scores on a held‑out diagnostic benchmark, with all weights, tokenizers, and code released openly.
The paper presents a corpus of 1,262 Classical Tamil verse‑commentary pairs and evaluates several neural representation learning models—including recurrent, Transformer, Siamese, mBART‑style encoder‑decoder, and decoder‑only language models—against a TF‑IDF baseline. Experiments reveal limited gains: token‑F1 scores range from 0.02 to 0.20, the encoder‑decoder continues to lower training loss even after validation loss rises, and the decoder‑only model only reproduces authentic word order in 95.5% of minimal‑pair tests but fails to generate held‑out commentary content. The authors release the extraction and evaluation protocol while noting that redistribution of the source commentaries requires permission.
MoganColBERT-TR is a Turkish multi‑vector retrieval model that projects token embeddings from 768 to 128 dimensions and uses MaxSim late interaction for scoring. It builds on the previously trained MoganBERT‑TR encoder, adapting it to the ColBERT objective with a single‑epoch distillation phase using cross‑encoder teacher scores. Evaluated on five Turkish BEIR datasets in a zero‑shot setting, it achieves an overall score of 37.36, outperforming larger models on most datasets.
arXiv:2607.15209v4 Announce Type: replace Abstract: Multilingual pre-trained language models such as XLM-R perform well for major languages but struggle with low-resource Ge'ez-script languages, larg...
Yalda-Embedding is a 595.8‑million‑parameter Persian retrieval model built by adapting Qwen3‑Embedding‑0.6B and trained on over one million query‑document pairs. It achieves the highest macro‑average score (0.5488) among 0.6B models on eight FaMTEB Persian retrieval tasks, outperforming Qwen3‑Embedding‑0.6B and F2LLM‑v2‑0.6B on every task and approaching the performance of the larger Qwen3‑Embedding‑4B. The authors also conduct an ablation study on contrastive batch size, optimization stabilization, and relational knowledge distillation, showing that the distilled configuration improves all tasks without increasing inference‑time model size and includes hard‑negative evaluation gains despite no explicit hard‑negative mining.
We present DONDO, a family of open, permissively licensed automatic speech recognition (ASR) base models for African languages, built on the w2v-BERT 2. 0 self-supervised speech encoder.