arXiv:2605. 24818v2 Announce Type: replace-cross Abstract: The literature on test set contamination largely focuses on detection, but the correction of contaminated test scores is underexplored.
The paper investigates how benchmark contamination—leakage of test items into training data—affects large language model (LLM) leaderboards. By comparing original test items with semantically equivalent paraphrases, the authors measure contamination as a violation of anchor-item invariance and find that it inflates absolute scores but rarely changes model rankings. Across 47 public models and 74 finetuned models on four benchmarks, the rank correlation between standard and paraphrase-controlled leaderboards is 0.997, with only a handful of cases showing differential contamination that could alter rankings.
arXiv:2606. 31208v1 Announce Type: new Abstract: Large tabular models (LTMs), i.
arXiv:2510. 20351v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly exposed to data contamination, i.
arXiv:2608. 02985v1 Announce Type: new Abstract: The standard check for contamination in LLM backtests is simple: compare scores before and after the training cutoff.
The paper uncovers a hidden flaw in evaluating machine unlearning on BatchNorm-based models: a single forward pass over retained data can alter the model’s normalization state without changing weights, misleadingly restoring performance. This effect, formalized as a weight‑preserving fixed‑point operator, shows that apparent forgetting can be entirely due to BatchNorm running statistics, not to the unlearning method itself. Experiments demonstrate that the artifact can inflate forget accuracy by up to 78 percentage points across nine methods, and that switching to GroupNorm eliminates the problem.
arXiv:2402. 08922v3 Announce Type: replace Abstract: Large-scale black-box models have become ubiquitous across numerous applications.
arXiv:2608.27704v1 Announce Type: new Abstract: When machine learning classifiers are retrained, inputs correctly classified by the previous model version may be misclassified by the updated version,...
arXiv:2607. 12649v1 Announce Type: new Abstract: Recent work on extractable memorization in LLMs suffers from two contrasting validity problems.
arXiv:2509. 17314v4 Announce Type: replace-cross Abstract: Software increasingly relies on the emergent capabilities of Large Language Models (LLMs), from natural language understanding to program analysis and generation.
arXiv:2606. 04310v1 Announce Type: new Abstract: Deep Neural Networks (DNNs) are increasingly being deployed in security-critical and safety-sensitive applications, which makes rigorous testing essential to identify and mitigate model weaknesses.
arXiv:2606. 25001v1 Announce Type: new Abstract: Machine unlearning (MU) is commonly judged by output forgetting, such as low forget-set accuracy or reduced logit-level membership inference.