Large language models can reveal memorized training sequences when prompted with a prefix, but real-world systems often use additional context such as instructions or retrieved documents. This study measures how such contextual conditioning affects memorization by comparing suffix probabilities with empty prompts versus retrieved contexts across three instruction-tuned models. The results show that context does not simply erase memorization; instead, it preserves a core set of robustly extractable samples while also shifting the boundary of marginal cases, sometimes suppressing and sometimes enabling new exposures.
arXiv:2602. 18733v2 Announce Type: replace Abstract: Training data leakage from Large Language Models (LLMs) raises serious concerns related to privacy, security, and copyright compliance.
arXiv:2607. 12649v1 Announce Type: new Abstract: Recent work on extractable memorization in LLMs suffers from two contrasting validity problems.
The paper explores ways to reduce the memorization of training data in language models, testing three regularizer-based, three finetuning-based, and eleven machine unlearning methods—five of which are newly introduced. It introduces TinyMem, a lightweight suite of small models for rapid testing of these mitigation techniques, and shows that unlearning methods, particularly BalancedSubnet, outperform others in removing memorized content while maintaining task performance. The study also finds that regularizer-based approaches are slow and ineffective, while finetuning methods are costly, especially when high accuracy is required.
arXiv:2606. 06286v1 Announce Type: cross Abstract: Large language models can reproduce training data, but existing memorization evaluations mostly measure whether models can be forced to do so, rather than whether they do so under ordinary use.
arXiv:2606. 06320v1 Announce Type: new Abstract: Machine unlearning aims to remove targeted knowledge from a trained model while preserving its general capabilities.
The paper introduces Selective Prefix Anti-Interference Regularization (SPAR), a pretraining objective that trains language models to ignore irrelevant distant prefixes. SPAR compares predictions from the original sequence with those from a corrupted prefix that only alters far tokens, using a sufficiency gate and a gated KL loss to stabilize suffix predictions that are already locally supported. Experiments show that SPAR improves robustness metrics such as RULER and NoLiMa across several models, including Qwen2.5, Llama, and GPT2-XL, under equal compute.
The paper evaluates open‑source large language models (LLMs) for both flat and nested named entity recognition (NER) tasks, comparing them to traditional encoder‑based models and decoder‑based LLMs using in‑context learning. It shows that with parameter‑efficient fine‑tuning and structured output formats (e.g., inline brackets or XML), LLMs can match or exceed the performance of conventional NER systems, and that their NER abilities arise from instruction following rather than memorization. Additionally, fine‑tuning for NER minimally affects, and can even improve, the LLMs’ general capabilities on other benchmarks.
The paper introduces ADU, a fine‑grained training framework that unlearns sensitive information from large language models by decoupling contextual attention pathways instead of erasing tokens. ADU exploits the distinction between local and global attention heads to identify and suppress attention paths that retrieve persistent sensitive anchors, while preserving local‑attention structure and overall language modeling performance. Evaluation on the TOFU and WMDP benchmarks shows ADU achieves superior forget quality (0.93 on TOFU) and retains 92.9% of model utility compared to 81.9% for existing baselines, with fewer side effects in benign contexts.
The paper introduces DRY, a sampling-time logit adjustment that penalizes token generation only when it would extend the current suffix into an exact repetition of an earlier span, thereby preventing verbatim loops in large language model outputs. Experiments across models ranging from 1.5B to 120B parameters and various prompt families show that DRY cuts suffix-extension rates by 47% and improves lexical diversity, while preserving benchmark performance. The method has been adopted by popular open-source LLM inference frameworks, indicating its practical relevance.
arXiv:2607. 09236v1 Announce Type: new Abstract: Machine unlearning in LLMs is the targeted removal of specific knowledge while preserving all other capabilities, critical for privacy and safety.
arXiv:2603. 24917v2 Announce Type: replace-cross Abstract: Recent work shows that standard greedy-decoding extraction methods for quantifying memorization in LLMs miss how extraction risk varies across sequences.
CoEM introduces a Commit-on-Evidence Memory system that learns when to compress source evidence into compact memory facts while preserving potentially useful excerpts verbatim in a pending set. The system uses a learned policy to decide whether to promote, retain, or discard each pending excerpt as new context arrives, and a frozen verifier ensures only supported facts are committed. Reinforcement learning trains this policy with step-level evidence rewards and final answer rewards, leading to consistent improvements in long-context reasoning, achieving 10.4–11.4 F1 points over the strongest baseline on 6,400-document inputs.