← Back to all news
arXiv Machine Learning By Arman Nik Khah

Detectable Only Where It Is Confounded: What Verified Duplication Counts Say About Membership Evidence in Language Models

Read the original on arXiv Machine Learning →

The paper investigates whether language models can identify sentences from their training data by using exact duplication counts from publicly released corpora for two model families, OLMo‑2 and Pythia. It finds that for typical duplication levels, models show only a weak trace of exposure, with a rank correlation near –0.08, and that strong signals only appear when a sentence appears roughly a thousand times, at which point fame rather than memory dominates. The study also demonstrates that common membership tests can be misleading, as changing a single word does not alter the model’s preference, and that controlling for register can significantly improve detector performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Computation and Language
Sep 25

What a Cross-Model Fixed-Point Census Can and Cannot Arbitrate About Repetition

The paper investigates neural text degeneration by measuring the fixed‑point structure of short‑window argmax maps across 17 pretrained models, using 96 random two‑token starts without prompts. It finds a stable four‑way classification that varies across model families and scales, with some models funneling to a single endpoint token while others do not, and shows that this behavior is not solely determined by training data or corpus frequency. The study demonstrates that repetition phenomena are not uniformly explained by either training data or network architecture alone, highlighting the complexity of neural text generation dynamics.

By Nicol\'as Vera Z\'u\~niga
llms
More like this →
arXiv Machine Learning
Oct 3

When a Data Artifact Isn't a Shortcut: Causal Auditing of Synthetic RLVR Corpora

The paper audits whether synthetic distractors in RLVR corpora act as shortcuts for learning policies. A classifier using only surface statistics barely outperforms chance, and manual inspection reveals that code distractors are almost identical to correct answers. Experiments with a paraphrase‑matched control show no exploitation advantage for the unmodified data, indicating that the detectable artifact was not used by the policy.

By Esther Xin
llms
More like this →
arXiv AI
Oct 16

Language models recognize dropout and Gaussian noise applied to their activations

Language models can detect, localize, and even verbalize differences between dropout and Gaussian noise applied to their activations. In experiments on eight models ranging from 8 B to 36 B, many achieve near‑perfect accuracy in identifying which sentence was perturbed and which type of perturbation was used. The study shows that this ability emerges during pre‑training, correlates with perturbation strength, and can be learned in context, while incorrect labels degrade performance.

By Damiano Fornasiere, Mirko Bronzi, Spencer Kitts, Alessandro Palmas, Yoshua Bengio, Oliver Richardson
llmssafety
More like this →