arXiv AI

Beyond Cross-Lingual Transfer: Benchmarking Propagation Boundaries in Multilingual LLM Unlearning

The paper introduces CLLPU, a multilingual benchmark for evaluating how well large language models can unlearn specific knowledge while controlling its propagation across languages. CLLPU defines two forgetting scenarios—common-goal forgetting, which requires suppression across all languages, and language-conditioned forgetting, which limits suppression to a single language. Using 800 knowledge-unit pairs and 72,000 QA instances in ten languages, the authors test six methods on Llama‑3.1‑8B‑Instruct and find that universal suppression often fails, while language‑specific suppression can unintentionally spread to other languages, highlighting the difficulty of propagation control in multilingual unlearning.

arXiv AI
Oct 1

Linguistic Loopholes in LLM Unlearning: From a 174-Language Benchmark to Coverage-Aware Unlearning

The paper introduces the problem of cross‑lingual loopholes in large language model (LLM) unlearning, where forgetting a fact in one language can leave it accessible in others. It presents a new 174‑language benchmark, the Cross‑Lingual Unlearning Tensor, and proposes COVER, a method that selects a subset of source languages to maximize unlearning coverage under a language budget. Experiments show COVER reduces residual knowledge by 7.8–27.3% compared to uniform selection and works on both synthetic and real low‑resource news data.

By Tyler Skow, Shravan Chaudhari, Rama Chellappa, Abhay Yadav
arXiv Computation and Language
Oct 9

Not Every Change Is Necessary: Recoverable Drift in Large Language Model Unlearning

The paper introduces Propose-Then-Project Unlearning (PTP-U), a method for large language model unlearning that first weakens target knowledge associations and then projects non-target outputs back to the original distribution to recover capabilities. PTP-U aims to satisfy forgetting constraints while preserving fluent generation and performance on non-target tasks. Experiments on three benchmarks show PTP-U achieves a strong forgetting‑retention trade‑off, with 81.22%–91.03% forgetting and an average of 94.20% non‑target utility.

By Xunlei Chen, Qinghui Gong, Jingkun Xue, Qihe Liu, Shijie Zhou, Fei Ye
arXiv Computation and Language
Sep 10

MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages

arXiv:2607.00890v2 Announce Type: replace Abstract: Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synth...

By Maximilian Idahl, J\"org Tiedemann, Sampo Pyysalo, David Salinas, Tomasz Galica, Shenbin Qian, Tudor Nicolae Mateiu, Zihao Li, Anna Lokrantz, Fedor Vitiugin, Andr\'e F. T. Martins, Jenna Kanerva, Filip Ginter, Matthias Lindemann, Tim Isbister, Birger Moell, Jonas Lindh, Jan Haji\v{c}, Jenia Jitsev, Andrey Kutuzov, Stephan Oepen, Gema Ram\'irez-S\'anchez
arXiv AI
Sep 4

One Model to Translate Them All? A Journey to Mount Doom for Multilingual Model Merging

The paper investigates weight‑space merging of independently fine‑tuned multilingual machine translation models. Experiments show that merging is more successful when models share a target language, yet it still cannot match the peak performance of language‑specific checkpoints. When target languages differ, performance drops sharply, and analysis reveals that overlapping neuron activation and incompatible upper‑layer geometries cause these failures.

By Baban Gain, Trilok Nath Singh, Asif Ekbal
arXiv Computation and Language
Aug 28

Cross-lingual Representation Learning via Centroid Intervention Fusion

The paper introduces Centroid Intervention Fusion (CIF), a framework that merges multiple multilingual intervention projections into a single language-shared operator for inference-time modification of large language models. CIF improves cross-lingual transfer without updating model parameters and achieves up to +3.378 percentage points better performance than prior pairwise intervention baselines across several benchmarks, including low-resource languages. The authors provide code at https://github.com/VRCMF/CIF.git.

By Wei Sun, Marie-Francine Moens