The Flow has not summarised this story yet — read it at arXiv Computation and Language.
The study evaluates how different input representations—orthographic text, IPA transcription, and romanization—affect cross‑lingual transfer in autoregressive multilingual language models. Across three model sizes and eight languages grouped into typologically motivated pairs, romanized pretraining consistently outperforms native orthography and IPA, especially as model scale increases. Fine‑tuning a text‑pretrained model on romanized data can harm performance on languages already covered by the base model, suggesting romanization should be integrated at pretraining rather than applied later.
arXiv:2608. 07629v1 Announce Type: cross Abstract: Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon.
Low-resource languages remain challenging for machine translation, and Mongolian is a representative case. As a digraphic language, Mongolian is written in both Cyrillic and Traditional scripts, which exhibit a severe imbalance in data availability.
arXiv:2609.39640v1 Announce Type: cross Abstract: Cross-lingual transfer describes how knowledge in a source language benefits a target language. Measuring it quantitatively requires broad multilingu...
Pixel language models treat text as rendered images, enabling strong cross‑lingual transfer. This study trains MPIXEL models varying languages, scripts, and capacity while keeping per‑language compute constant, and evaluates them on five tasks across 83 languages. Results show that scaling improves performance on added languages and scripts, mitigates the curse of multilinguality, and outperforms subword and character baselines on typologically distant low‑resource languages, though transfer to unseen scripts does not benefit further from scaling.
arXiv:2510. 07074v2 Announce Type: replace-cross Abstract: Instruction tuning has become a key technique for enhancing the performance of large language models, enabling them to better follow human prompts.