arXiv Computation and Language

Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM

arXiv:2609. 17435v1 Announce Type: new Abstract: We submit M\'eTRON-FR, a 125M GPT-2 pretrained on 92.

arXiv Computation and Language
Sep 28

Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders

The paper challenges the assumption that improving cross‑lingual alignment automatically enhances cross‑lingual transfer. Using XLM‑R models aligned on token, sentence, and masked‑language‑modeling objectives across four language pairs, the authors evaluate zero‑shot transfer on part‑of‑speech tagging and sentence classification. They find that embedding‑based alignment metrics poorly predict downstream performance and that alignment and task gradients are often nearly orthogonal, especially when operating at different representational levels.

By Yana Veitsman, Yihong Liu, Hinrich Sch\"utze
arXiv Computation and Language
Aug 25

LuxIT: A Luxembourgish Instruction Tuning Dataset from Monolingual Seed Data

LuxIT is a monolingual instruction‑tuning dataset for Luxembourgish, created by synthesizing instruction‑answer pairs from native texts using the DeepSeek‑R1‑0528 model and a quality‑assurance LLM‑as‑judge process. The resulting 227,507 high‑quality pairs were used to fine‑tune 14 LLMs (≤15 B parameters), yielding an average accuracy increase of +5.37 percentage points on standardized Luxembourgish proficiency exams and improvements in macro‑averaged F1 on nine of the fourteen downstream NLP tasks. These findings demonstrate that synthetic monolingual data can effectively enhance LLM performance in low‑resource languages and reveal the complex relationship between exam performance and practical NLP gains.

By Julian Valline, Cedric Lothritz, Siwen Guo, Jordi Cabot
arXiv Machine Learning
Aug 5

M-GATE: Multilingual Grammar, Accuracy in Translation, and Efficiency Benchmark for Large Language Models

arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.

By Tom\'a\v{s} Burkert, Angelika Peljak-{\L}api\'nska, David Zelen\'y
arXiv Computation and Language
Sep 7

EuroAlpaca: Task-Preserving Localisation of Instruction Data for European Languages

EuroAlpaca presents a task‑preserving localisation pipeline that translates English instruction‑tuning data into 50 European languages while maintaining task‑critical constraints. The method uses field‑wise machine translation or reconstructs task‑equivalent target‑language instances, followed by validation of coherence and consistency. Experiments show that EuroAlpaca improves instruction‑following accuracy by 12.9% over a baseline and outperforms direct translation on ROUGE‑L and F‑BERT metrics.

By Aleix Sant, Jordi Luque, Carlos Escolano
arXiv Computation and Language
Oct 12

Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining

Large language models show strong cross‑lingual abilities, but previous studies have examined them only at isolated training points. This work pretrains a 1.7B multilingual model on nine languages, recording checkpoints at fine granularity to track translation development. Using word‑level translation and a new dataset, the authors find that early learning is dominated by token‑level copying and surface similarities, while later stages involve more general translation mechanisms and refined copying.

By Felicia K\"orner, Maria Matveev, Florian Eichin, Gitta Kutyniok, Barbara Plank, Michael A. Hedderich
arXiv Computation and Language
Nov 6

Different Corpora, Different Skills? A Paired Study of Small Language Models

The study compares two small‑scale pretraining corpora—1911 Encyclopaedia Britannica and BabyLM 2026—using matched 10‑million and 37‑million‑word budgets. Models trained on BabyLM 2026 consistently outperform those trained on Britannica on linguistic acceptability tasks, while Britannica models excel at Entity Tracking, a synthetic state‑tracking benchmark. These divergent skill profiles appear across teacher and student models, both budgets, and persist even after controlling for training exposure, indicating that corpus choice can shape relative strengths in small language models.

By Edoardo Vaira, Georgios Chavales, Pierre Nugues
arXiv Computation and Language
Oct 8

sk-bench: A Native-First Benchmark for Evaluating Large Language Models in Slovak

arXiv:2610.09152v1 Announce Type: new Abstract: Multilingual LLM benchmarks omit Slovak, a morphologically rich West Slavic language of five million speakers, or cover it only by machine translation....

By Marek \v{S}uppa, Ivan Vykopal, Andrej Ridzik, Kristi\'an Sopkovi\v{c}, Nat\'alia K\v{n}a\v{z}ekov\'a, Jaroslav Kop\v{c}an, Miroslav Bl\v{s}t\'ak, Vikt\'oria Ondrejov\'a, Daniel Hl\'adek, Michal Gregor, Martin Tamajka, Mari\'an \v{S}imko
arXiv Computation and Language
Sep 10

MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages

arXiv:2607.00890v2 Announce Type: replace Abstract: Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synth...

By Maximilian Idahl, J\"org Tiedemann, Sampo Pyysalo, David Salinas, Tomasz Galica, Shenbin Qian, Tudor Nicolae Mateiu, Zihao Li, Anna Lokrantz, Fedor Vitiugin, Andr\'e F. T. Martins, Jenna Kanerva, Filip Ginter, Matthias Lindemann, Tim Isbister, Birger Moell, Jonas Lindh, Jan Haji\v{c}, Jenia Jitsev, Andrey Kutuzov, Stephan Oepen, Gema Ram\'irez-S\'anchez
arXiv Computation and Language
Aug 28

Double Trouble: Bilingual Pretraining Leaves Language-Conditioned Effects in Shared-Language Representations

The study compares an English-only and a bilingual decoder-only model, each 310 M parameters, trained on eight diverse languages while controlling for English exposure, compute, and document overlap. After aligning on shared English vocabulary, the authors find that token embeddings appear similar, but the deeper hidden states used for prediction diverge across models. This hidden‑state mismatch grows through middle transformer layers and persists despite controls, indicating that contextual processing differs between the models. "whyItMatters":"The findings show that embedding alignment can conceal significant internal representation differences, which is crucial for any downstream work that assumes aligned multilingual models are interchangeable."

By Anjishnu Mukherjee, Ziwei Zhu, Antonios Anastasopoulos
arXiv Computation and Language
Sep 30

Cross-Linguistic Effects in Bilingual Phoneme BabyLMs

arXiv:2609.37121v1 Announce Type: new Abstract: Cross-linguistic effects are a central topic in bilingual first-language acquisition. Artificial learners can help investigate L1-L2 interactions by en...

By Nikitas Theodoropoulos, Maria Lymperaiou, Giorgos Filandrianos
arXiv Computation and Language
Oct 22

Task-Preserving Multilingual Instruction Tuning for Open-Ended QA in WMT 2026 MIST

The paper reports a submission to the WMT 2026 Multilingual Instruction Shared Task, where the authors fine‑tune Qwen3‑8B with LoRA on organiser‑provided data, EuroAlpaca, and CrossEuroAlpaca. Their system focuses on open‑ended question answering in ten target languages and shows a 7.8 % relative gain in Gemma‑4 judge score for target‑language QA‑OEG, as well as 41.5 % macro‑averaged ROUGE‑L and 2.9 % BERTScore‑F1 improvements on the Aya Evaluation Suite. However, the gains are task‑ and language‑specific, with some declines in context‑grounded QA and non‑target languages.

By Aleix Sant, Jordi Luque, Carlos Escolano