The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2604.13286v2 Announce Type: replace Abstract: Despite the widespread multilingual deployment of large language models, post-training pipelines remain predominantly English-centric, contributing...
arXiv:2410. 07809v2 Announce Type: replace-cross Abstract: Multilingual instruction tuning (MIT) is challenged by the curse of multilinguality, data scarcity, and high computational cost.
The paper investigates how many languages should be jointly trained in a single lexical normalization model. Using a fixed-capacity character-level model across twelve languages, it finds that accuracy peaks when a language is trained with only a few others—typically one to four—and then declines sharply as more languages are added, dropping about forty percent. A control experiment keeping total training data constant shows the decline is due to competition for model capacity rather than data scarcity, and no reliable typological rule predicts the optimal number of co‑training languages.
arXiv:2608. 04586v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) have achieved significant success in speech-to-text translation (S2TT).
The study investigates how the choice of tokenizer influences multilingual language models across 54 tokenizers and 123 models. It finds that tokenizer impact is greater for languages with less training data, and that excluding a language from tokenizer training consistently worsens its performance. While reallocating tokenizer training data to lower-resource languages can help, it does not guarantee improvement, and the best tokenizer properties vary by language, enabling predictive screening of tokenizer candidates.
arXiv:2608. 14071v1 Announce Type: new Abstract: As large language models scale, their training-token budgets must also increase to maintain an appropriate tokens-per-parameter ratio (\(\mathrm{TPP}\)).