The Flow has not summarised this story yet — read it at arXiv Computation and Language.
arXiv:2609.16340v1 Announce Type: cross Abstract: Machine translation systems are periodically upgraded to stronger models, but the available preference signal is human post-edits of an older system'...
The paper introduces the Last Translation Benchmark (LTB), a live dataset of human-authored and peer‑reviewed examples—including texts, images, audio, and videos—that are designed to break current state‑of‑the‑art machine translation models. Each example is accompanied by handcrafted verification rules that specify concrete failure cases, providing a reliable and actionable evaluation method. The benchmark aims to overcome the limitations of existing automatic metrics and gold human evaluations, which often lack reproducibility, objectivity, and scalability.
The paper introduces ProNMT, a reward-guided iterative self‑training approach that balances global translation quality with pronoun‑specific feedback for context‑aware machine translation. ProNMT samples candidate translations, scores them using reference‑free quality estimation and a pronoun label derived from references, and fine‑tunes on the highest‑scoring candidate. Experiments on English–German Europarl and English–French News Commentary show that ProNMT outperforms standard context‑aware fine‑tuning on BLEU and COMET, while ablations reveal that pronoun‑only feedback can harm overall quality and that confidence‑weighted feedback outperforms hard binary feedback.
As state-of-the-art machine translation models saturate standard benchmarks, the field needs more challenging evaluations to distinguish between models of varying quality. We propose augmenting existi...
arXiv:2608. 15932v1 Announce Type: new Abstract: As state-of-the-art machine translation models saturate standard benchmarks, the field needs more challenging evaluations to distinguish between models of varying quality.
EuroAlpaca presents a task‑preserving localisation pipeline that translates English instruction‑tuning data into 50 European languages while maintaining task‑critical constraints. The method uses field‑wise machine translation or reconstructs task‑equivalent target‑language instances, followed by validation of coherence and consistency. Experiments show that EuroAlpaca improves instruction‑following accuracy by 12.9% over a baseline and outperforms direct translation on ROUGE‑L and F‑BERT metrics.