arXiv Computation and Language
Sep 28

Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders

The paper challenges the assumption that improving cross‑lingual alignment automatically enhances cross‑lingual transfer. Using XLM‑R models aligned on token, sentence, and masked‑language‑modeling objectives across four language pairs, the authors evaluate zero‑shot transfer on part‑of‑speech tagging and sentence classification. They find that embedding‑based alignment metrics poorly predict downstream performance and that alignment and task gradients are often nearly orthogonal, especially when operating at different representational levels.

By Yana Veitsman, Yihong Liu, Hinrich Sch\"utze