Introducing The World's Largest Open Multilingual Language Model: BLOOM

A Deepdive into Aya Expanse: Advancing the Frontier of Multilinguality

Understanding the capabilities, limitations, and societal impact of large language models

No Optimal Language Set Exists for Multilingual Instruction Tuning: Insights from a Linguistically-Informed Study

arXiv:2410. 07809v2 Announce Type: replace-cross Abstract: Multilingual instruction tuning (MIT) is challenged by the curse of multilinguality, data scarcity, and high computational cost.

The Curse of Multilinguality in Lexical Normalization

The paper investigates how many languages should be jointly trained in a single lexical normalization model. Using a fixed-capacity character-level model across twelve languages, it finds that accuracy peaks when a language is trained with only a few others—typically one to four—and then declines sharply as more languages are added, dropping about forty percent. A control experiment keeping total training data constant shows the decline is due to competition for model capacity rather than data scarcity, and no reliable typological rule predicts the optimal number of co‑training languages.

Setting Up Your Own Large Language Model

Still a long way to go, but the future is promising The post Setting Up Your Own Large Language Model appeared first on Towards Data Science .

$\mu^2$-Bench: A Multilingual Machine Unlearning Benchmark

arXiv:2609. 20945v1 Announce Type: new Abstract: Undesired information such as harmful content and private data propagates through Multilingual Large Language Models (LLMs) via direct training and indirect cross-linguistic spread.

Very Large Language Models and How to Evaluate Them

Red-Teaming Large Language Models

Cross-lingual Representation Learning via Centroid Intervention Fusion

The paper introduces Centroid Intervention Fusion (CIF), a framework that merges multiple multilingual intervention projections into a single language-shared operator for inference-time modification of large language models. CIF improves cross-lingual transfer without updating model parameters and achieves up to +3.378 percentage points better performance than prior pairwise intervention baselines across several benchmarks, including low-resource languages. The authors provide code at https://github.com/VRCMF/CIF.git.

One Model to Translate Them All? A Journey to Mount Doom for Multilingual Model Merging

The paper investigates weight‑space merging of independently fine‑tuned multilingual machine translation models. Experiments show that merging is more successful when models share a target language, yet it still cannot match the peak performance of language‑specific checkpoints. When target languages differ, performance drops sharply, and analysis reveals that overlapping neuron activation and incompatible upper‑layer geometries cause these failures.

Multilinguality of Large Language Models From a Structural Perspective

arXiv:2606. 01800v1 Announce Type: cross Abstract: Large language models (LLMs) have excelled in processing multiple languages through pre- and post-training on multilingual data, even though English dominates the training data.

Beyond the Rosetta Stone: Unification Forces in Generalization Dynamics

The paper investigates why large language models sometimes hallucinate when asked about facts in a language different from the one in which the facts were learned. By training small Transformer models on synthetic multilingual datasets, the authors show that the degree of correlation between facts and their learning language (informativeness) and the ease of language identification (extractability) determine whether models develop unified or separate representations across languages. Unified representations enable cross‑lingual fact transfer, while separate representations do not. The study proposes a unifying perspective on cross‑lingual transfer and suggests training methods to promote representational unification.