The study investigates how exposing large language models to Arabic translations of English benchmark tasks can hide data contamination from standard English-only detection methods. By testing four instruction-tuned models with increasing Arabic exposure, the authors find that traditional probes like TS-Guessing and Min-K%++ fail to detect contamination, yet English performance still improves. They propose a new Translation-Aware Contamination Detection (TACD) method that uses cross-lingual consistency and choice reordering to reveal contamination effects that are otherwise invisible to English-only diagnostics.
By Chaymaa Abbas, Nour Shammaa, Mariette Awad
arXiv:2606. 03305v1 Announce Type: new Abstract: Benchmark contamination, where evaluation examples appear in a model's training data, threatens the validity of LLM assessment.
By Wojciech Zarzecki, Jan Dubi\'nski, Sebastian Cygert
arXiv:2604. 01904v3 Announce Type: replace-cross Abstract: Post-hoc unauthorized-training data detection for large language models (LLMs) typically assumes a query-with-originals regime: rights holders query a target LLM with raw proprietary data and assess whether the model assigns them stronger memorization-based detection signals, e.
By Muxing Li, Zesheng Ye, Sharon Li, Feng Liu
arXiv:2608.21230v1 Announce Type: cross
Abstract: Persistent memory makes false information durable: once a false statement is stored, it can be retrieved into future sessions that match it. We measu...
By Arulnidhi Karunanidhi
arXiv:2608. 16643v1 Announce Type: cross Abstract: Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such models rest on benchmarks that evaluate each clinical note in isolation.
By Yifan Zhang, Rahmatollah Beheshti
The paper investigates how benchmark contamination—leakage of test items into training data—affects large language model (LLM) leaderboards. By comparing original test items with semantically equivalent paraphrases, the authors measure contamination as a violation of anchor-item invariance and find that it inflates absolute scores but rarely changes model rankings. Across 47 public models and 74 finetuned models on four benchmarks, the rank correlation between standard and paraphrase-controlled leaderboards is 0.997, with only a handful of cases showing differential contamination that could alter rankings.
By Xingyao Xiao (Stanford University), Yihong Cheng (City University of Macau)
The study examines how the rapid turnover of large language models (LLMs) undermines the effectiveness of AI‑detectors used to screen scientific manuscripts. By comparing 4,000 PNAS abstracts rewritten by 23 different LLM versions released between June 2023 and August 2026, the authors show that detectors trained on past versions can fail dramatically at generation boundaries, missing up to 96% of rewrites just after a sharp transition. Even detectors updated to newer versions can overlook rewrites of older models, and a commercial detector missed most rewrites of the newest version while flagging few human‑written abstracts.
By Kazuki Nakajima, Takayuki Mizuno
arXiv:2607. 26339v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) systems ground large language models (LLMs) in external corpora, but this reliance exposes them to corpus poisoning: maliciously injected passages that manipulate retrieved evidence.
By Pushkal Kumar, Tucker Nielson, Tanish Kolhe, Shubham Zala, Vincent Li
Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such models rest on benchmarks that evaluate each clinical note in isolation. Error-detection benchmarks are typically constructed by injecting errors into notes, such that each erroneous note has a natural counterpart.
arXiv:2608. 09209v1 Announce Type: cross Abstract: Neural language models trained on large crowdsourced corpora frequently exploit spurious surface patterns tied to target labels without true linguistic or causal relevance, boosting benchmark performance while failing on adversarial or out-of-distribution inputs.
By Chidaksh Ravuru, Shashank Srivastava
The paper introduces SimpleLoRIF, a scalable gradient‑embedding technique that pinpoints fine‑tuning examples whose gradients align with unsafe behavior in large language models. By applying this method, the authors create SafeTulu, a version of Tulu 3 with the 20 % of most unsafe‑aligned examples removed, which reduces GCG attack success by up to 22 % across several LLMs while largely preserving or improving capability. The study also highlights previously under‑studied safety‑degrading data sources such as tool‑execution failures and narrative arithmetic.
By Saket Reddy, Amro Abbas, Anshuman Suri, Ari Morcos, David Schwab, Pratyush Maini
The study investigates whether provenance information can reliably identify the source of synthetic text and whether this identification improves the selection of training data. Using financial‑risk text, the authors achieve 98.7% accuracy in attributing original generated passages, but accuracy drops to 53.1% after paraphrasing and 29.0% after style rewriting. They compare two selection strategies—one based on source provenance and another on a reference model score—across three rounds of generation and retraining, finding that the two methods choose different examples but do not produce a consistent difference in model degradation. The results suggest that source attribution and useful data selection are distinct challenges, and neither provenance nor the tested proxy suffices to guarantee stable recursive training behavior.
By Joss Armstrong