arXiv Computation and Language By Adam Zachary Wasserman, David Beauchemin

Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM

Read the original on arXiv Computation and Language →

arXiv:2609. 17435v1 Announce Type: new Abstract: We submit M\'eTRON-FR, a 125M GPT-2 pretrained on 92.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Computation and Language
Sep 28

Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders

The paper challenges the assumption that improving cross‑lingual alignment automatically enhances cross‑lingual transfer. Using XLM‑R models aligned on token, sentence, and masked‑language‑modeling objectives across four language pairs, the authors evaluate zero‑shot transfer on part‑of‑speech tagging and sentence classification. They find that embedding‑based alignment metrics poorly predict downstream performance and that alignment and task gradients are often nearly orthogonal, especially when operating at different representational levels.

By Yana Veitsman, Yihong Liu, Hinrich Sch\"utze
arXiv Computation and Language
Aug 25

LuxIT: A Luxembourgish Instruction Tuning Dataset from Monolingual Seed Data

LuxIT is a monolingual instruction‑tuning dataset for Luxembourgish, created by synthesizing instruction‑answer pairs from native texts using the DeepSeek‑R1‑0528 model and a quality‑assurance LLM‑as‑judge process. The resulting 227,507 high‑quality pairs were used to fine‑tune 14 LLMs (≤15 B parameters), yielding an average accuracy increase of +5.37 percentage points on standardized Luxembourgish proficiency exams and improvements in macro‑averaged F1 on nine of the fourteen downstream NLP tasks. These findings demonstrate that synthetic monolingual data can effectively enhance LLM performance in low‑resource languages and reveal the complex relationship between exam performance and practical NLP gains.

By Julian Valline, Cedric Lothritz, Siwen Guo, Jordi Cabot
arXiv Machine Learning
Aug 5

M-GATE: Multilingual Grammar, Accuracy in Translation, and Efficiency Benchmark for Large Language Models

arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.

By Tom\'a\v{s} Burkert, Angelika Peljak-{\L}api\'nska, David Zelen\'y
arXiv Computation and Language
Sep 7

EuroAlpaca: Task-Preserving Localisation of Instruction Data for European Languages

EuroAlpaca presents a task‑preserving localisation pipeline that translates English instruction‑tuning data into 50 European languages while maintaining task‑critical constraints. The method uses field‑wise machine translation or reconstructs task‑equivalent target‑language instances, followed by validation of coherence and consistency. Experiments show that EuroAlpaca improves instruction‑following accuracy by 12.9% over a baseline and outperforms direct translation on ROUGE‑L and F‑BERT metrics.

By Aleix Sant, Jordi Luque, Carlos Escolano
arXiv Computation and Language
Oct 12

Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining

Large language models show strong cross‑lingual abilities, but previous studies have examined them only at isolated training points. This work pretrains a 1.7B multilingual model on nine languages, recording checkpoints at fine granularity to track translation development. Using word‑level translation and a new dataset, the authors find that early learning is dominated by token‑level copying and surface similarities, while later stages involve more general translation mechanisms and refined copying.

By Felicia K\"orner, Maria Matveev, Florian Eichin, Gitta Kutyniok, Barbara Plank, Michael A. Hedderich
arXiv Computation and Language
Nov 6

Different Corpora, Different Skills? A Paired Study of Small Language Models

The study compares two small‑scale pretraining corpora—1911 Encyclopaedia Britannica and BabyLM 2026—using matched 10‑million and 37‑million‑word budgets. Models trained on BabyLM 2026 consistently outperform those trained on Britannica on linguistic acceptability tasks, while Britannica models excel at Entity Tracking, a synthetic state‑tracking benchmark. These divergent skill profiles appear across teacher and student models, both budgets, and persist even after controlling for training exposure, indicating that corpus choice can shape relative strengths in small language models.

By Edoardo Vaira, Georgios Chavales, Pierre Nugues