The paper explores ways to reduce the memorization of training data in language models, testing three regularizer-based, three finetuning-based, and eleven machine unlearning methods—five of which are newly introduced. It introduces TinyMem, a lightweight suite of small models for rapid testing of these mitigation techniques, and shows that unlearning methods, particularly BalancedSubnet, outperform others in removing memorized content while maintaining task performance. The study also finds that regularizer-based approaches are slow and ineffective, while finetuning methods are costly, especially when high accuracy is required.
By Mansi Sakarvadia, Aswathy Ajith, Arham Khan, Nathaniel Hudson, Caleb Geniesse, Kyle Chard, Yaoqing Yang, Ian Foster, Michael W. Mahoney
The paper introduces a token‑level log‑likelihood ratio to pinpoint where large language models memorize training sequences. It proposes a probe‑steer framework, instantiated as Gated Subspace Steering (GSS), that selectively corrects memorization‑relevant activations using the leading singular vectors of a gradient‑weighted activation matrix. Experiments on four benchmarks show GSS matches or surpasses state‑of‑the‑art memorization reduction methods while using 10–100× less compute.
By Xuanqi Zhang, Haoyang Shang, Xiaoxiao Li
arXiv:2602. 18733v2 Announce Type: replace Abstract: Training data leakage from Large Language Models (LLMs) raises serious concerns related to privacy, security, and copyright compliance.
By Trishita Tiwari, Ari Trachtenberg, G. Edward Suh
Large language models are typically trained under uniform token weighting, which allows frequent and low-information tokens to dominate learning and can increase the tendency to memorize surface-level...
The paper proposes an information-weighted cross‑entropy loss that rescales token contributions using TF‑IDF statistics, thereby emphasizing semantically informative tokens and down‑weighting ubiquitous ones. Experiments on five decoder‑only language models (1.1B–13B parameters) show consistent reductions in memorized substring length while maintaining perplexity and downstream performance. The method is architecture‑agnostic, adds less than 3% computational overhead, and can be integrated into existing training pipelines.
By Zhijian Li, Stefan Larson, Kevin Leach
arXiv:2606. 29563v1 Announce Type: cross Abstract: Large language models (LLMs) excel at complex tasks like question answering and summarization, thanks to their ability to handle long-context inputs.
By Shuvendu Roy, Mengyao Zhai, Hossein Hajimirsadeghi, Golnoosh Samei
arXiv:2602.07120v3 Announce Type: replace
Abstract: Language models (LMs) tend to memorize portions of their training data and emit verbatim spans. When the underlying sources are sensitive or copyri...
By Jacqueline He, Jonathan Hayase, Wen-tau Yih, Sewoong Oh, Luke Zettlemoyer, Pang Wei Koh
arXiv:2606. 06286v1 Announce Type: cross Abstract: Large language models can reproduce training data, but existing memorization evaluations mostly measure whether models can be forced to do so, rather than whether they do so under ordinary use.
By Gianluca Barmina, Peter Schneider-Kamp, Lukas Galke Poech
The paper introduces Forgetting Only What Matters via Unlearning Layers (FOM-UL), a layer‑selective unlearning framework for large language models. FOM-UL uses a forget‑to‑retain significance score to identify transformer layers that strongly influence the forget set while being insensitive to the retain set, allowing targeted updates that preserve most of the model. Experiments on TOFU, KnowUnDo, and MUSE-style benchmarks show that FOM-UL reduces residual memorization and maintains utility better than several baselines, even after 8‑bit and 4‑bit post‑training quantization, and it also limits recovery of forgotten content in adversarial prompt tests.
By Ravi Ranjan, Olivera Kotevska, Agoritsa Polyzou
Large Language Models (LLMs) can memorize and reproduce sensitive, copyrighted, or otherwise undesirable training content, creating privacy, safety, and regulatory concerns. Machine unlearning offers...
Large language models can reveal memorized training sequences when prompted with a prefix, but real-world systems often use additional context such as instructions or retrieved documents. This study measures how such contextual conditioning affects memorization by comparing suffix probabilities with empty prompts versus retrieved contexts across three instruction-tuned models. The results show that context does not simply erase memorization; instead, it preserves a core set of robustly extractable samples while also shifting the boundary of marginal cases, sometimes suppressing and sometimes enabling new exposures.
By Ali Satvaty, Narjes Sharafi, Jirui Qi, Suzan Verberne, Fatih Turkmen
arXiv:2610.19247v1 Announce Type: new
Abstract: Transformers are intuitively considered to be efficient at copying from their inputs. We actually show that the ability of LLMs to copy in context can...
By Nathan Godey, Yoav Artzi