arXiv Computation and Language
Sep 21

Benchmarking Gender Bias in Machine Translation Evaluation Metrics across Occupations

The paper investigates gender bias in machine translation evaluation metrics using an occupation-balanced subset of GAMBIT+ across seven English‑source language pairs, including a new German extension. It finds that masculine translations tend to receive higher scores and that biases align with stereotypical gender representations, though the strength varies by evaluator and language. The study highlights that assessing bias requires multiple dimensions beyond a single aggregate measure.

By Orfeas Menis Mastromichalakis, Giorgos Filandrianos, Wafaa Mohammed, Giuseppe Attanasio, Chrysoula Zerva
arXiv Computation and Language
Sep 1

Beyond "To whom it may concern": Tailoring Machine Translation to Audience and Intent

The paper investigates how machine translation can be tailored to specific audiences and intents, a capability enabled by large language models (LLMs). By systematically evaluating purpose-driven MT across 50 languages, 5 model sizes, and 8 text domains, the authors find that explicit instructions significantly improve translation adaptiveness, especially for informal domains, larger models, and higher-resource languages. They also show that traditional MT metrics often penalize adapted translations and that models can self-generate useful instructions from context, closing a large portion of the adaptiveness gap.

By Raphael Merx, Ekaterina Vylomova, Trevor Cohn
arXiv AI
Sep 2

The Interlingua Hypothesis: LLMs Translate via a Latent Task-agnostic Feature Space

The paper proposes the interlingua hypothesis, suggesting that large language models translate by encoding a source sentence into a latent, task‑agnostic feature space and then decoding a target sentence from that space. Three lines of evidence support this: (1) BLEU variance across language pairs is largely explained by language‑specific competences without pair‑specific interactions; (2) many model components influence both monolingual and translation tasks; and (3) fine‑tuning on monolingual data recovers most translation gains seen with aligned documents. These findings converge to support the hypothesis and point toward new ways to understand and improve LLM translation.

By Jacob Brinton, Jannik Brinkmann, Mark Crovella, Aaron Mueller