arXiv Computation and Language

When the Weakest Language Keeps Moving: ARMOR-DP for Multilingual BabyLM

arXiv AI
Oct 9

Obscuring Data Contamination Through Translation: Evidence from Arabic Corpora

The study investigates how exposing large language models to Arabic translations of English benchmark tasks can hide data contamination from standard English-only detection methods. By testing four instruction-tuned models with increasing Arabic exposure, the authors find that traditional probes like TS-Guessing and Min-K%++ fail to detect contamination, yet English performance still improves. They propose a new Translation-Aware Contamination Detection (TACD) method that uses cross-lingual consistency and choice reordering to reveal contamination effects that are otherwise invisible to English-only diagnostics.

By Chaymaa Abbas, Nour Shammaa, Mariette Awad
arXiv Computation and Language
Oct 16

PRISM: Transfer and Limits of Activation Interventions in Tool-Calling Language Models

PRISM is a framework that evaluates how activation interventions transfer across precision, languages, and model scales to mitigate unsafe tool‑calling behaviors in language models. By leveraging lower‑resource settings such as quantized NF4 labels and cross‑lingual transfer, PRISM demonstrates that alignment improvements can be achieved with minimal scenarios and that these gains transfer to held‑out languages and larger models. The study shows that auxiliary supervision from compressed models and multilingual data can effectively uncover and reduce hazardous tool‑calling behaviors across different model families.

By Giovana Kerche Bon\'as, Thales Sales Almeida, Marcelo da Silva Reis, Rodrigo Nogueira, Marcos Meideiros Raimundo
arXiv AI
Oct 15

XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity

arXiv:2605.05662v2 Announce Type: replace-cross Abstract: Current LLM safety benchmarks are predominantly English-centric and often rely on translation, failing to capture country-specific harms. Mor...

By Dasol Choi, Eugenia Kim, Jaewon Noh, Sang Seo, Eunmi Kim, Myunggyo Oh, Yunjin Park, Brigitta Jesica Kartono, Josef Pichlmeier, Helena Berndt, Sai Krishna Mendu, Glenn Johannes Tungka, \"Ozlem G\"ok\c{c}e, Suresh Gehlot, Katherine Pratt, Amanda Minnich, Haon Park
arXiv Machine Learning
Oct 8

Quad-State Safety Evaluation of Open-Weight Large Language Models on Non-Canonical Inputs

The paper introduces the Adversarial Surface-Form Robustness Dataset (ASRD), which contains 2,100 prompts across seven surface‑form families to test large language models on non‑canonical inputs. Five open‑weight models were evaluated on 10,500 responses, and a Quad‑State Evaluation Rubric categorized each response as harmful compliance, safe response, comprehension failure, or indeterminate. Results show that emoji and invisible Unicode variations produce few comprehension failures, while leetspeak, encoded wrappers, and hybrid transformations lead to high comprehension failure rates and varied harmful compliance percentages.

By Pavan Maddula