arXiv:2608. 03803v1 Announce Type: cross Abstract: Multilingual language models are deployed across a hundred or more languages, yet most benchmarks test whether a model can perform a task _in_ a language rather than whether it commands the language itself, conflating fluency with proficiency.
The study investigates how exposing large language models to Arabic translations of English benchmark tasks can hide data contamination from standard English-only detection methods. By testing four instruction-tuned models with increasing Arabic exposure, the authors find that traditional probes like TS-Guessing and Min-K%++ fail to detect contamination, yet English performance still improves. They propose a new Translation-Aware Contamination Detection (TACD) method that uses cross-lingual consistency and choice reordering to reveal contamination effects that are otherwise invisible to English-only diagnostics.
The paper introduces a black‑box, inference‑time diagnostic for low‑resource Automatic Post‑Editing (APE) that distinguishes whether poor performance is due to insufficient training data or inconsistent training signals. By varying an edit‑distance penalty and analyzing the resulting TER‑vs‑λ curve and confidence‑based constraint ordering, the authors identify two failure modes—Binary Collapse and Confident Miscalibration—across multiple language pairs. The diagnostic also suggests practical next steps, such as applying a static constraint for immediate accuracy gains, and the authors release new English‑Sinhala and English‑Tamil APE datasets with accompanying code.
The paper describes the redevelopment of a small neural machine translation system for Sicilian, using parallel text extracted from the bilingual journal Arba Sicula and a language‑agnostic sentence encoder for alignment. It compares training small Transformers from scratch with a pretrained NLLB model fine‑tuned via low‑rank adapters, and applies a staged reverse‑training recipe that improves translation into Sicilian more than from it. Results show the pretrained model outperforms the small Transformers, but a multilingual adapter can degrade performance on untrained language directions.
arXiv:2610.33749v1 Announce Type: new Abstract: As machine translation systems approach human-level quality, traditional evaluation methodologies struggle to detect subtle translation errors. We crit...
arXiv:2606. 28843v1 Announce Type: cross Abstract: Fine-tuning a large language model is a ubiquitous method for enhancing its capability on a specific downstream task.
arXiv:2607.02235v2 Announce Type: replace-cross Abstract: LLM-as-a-Judge has become the dominant evaluation paradigm for many natural language generation tasks (albeit mostly in English) due to short...
arXiv:2609.16340v1 Announce Type: cross Abstract: Machine translation systems are periodically upgraded to stronger models, but the available preference signal is human post-edits of an older system'...
arXiv:2609.14963v1 Announce Type: new Abstract: As large language models become capable translators of classical texts, a key challenge is deciding which outputs need expert review when no human refe...
arXiv:2607. 02235v1 Announce Type: cross Abstract: LLM-as-a-Judge has become the dominant evaluation paradigm for many natural language generation tasks, due to shortcomings of conventional metrics and high correlations with human judgment, albeit mostly in English.
The study investigates whether the language of a user’s instruction affects the safety verdicts of agent guards. By translating 1,220 labeled steps from an English benchmark into German, French, Turkish, Swahili, and Amharic while keeping all other data identical, the authors find that many open, small, and purpose‑built guards alter their verdicts when the instruction is translated, with larger effects for rarer languages. The impact varies with guard size, with smaller models showing greater loss of recall and larger models (e.g., Gemini 3.8 Flash) remaining largely unchanged.
arXiv:2608. 05163v1 Announce Type: cross Abstract: A common assumption holds that switching to a non-English language makes a multilingual RAG system easier to attack for personal information.