The paper investigates what a service must retain to faithfully reproduce a training pipeline after data deletion requests, focusing on a rule where earlier similar records block later ones. It shows that a limit on future deletions precisely determines which records can still be needed for training, enabling an exact repair of training statistics after repeated deletions. The authors also construct finite‑precision features for ridge regression and demonstrate that the required private information can be minimized across a sequence of deletion requests, validating their approach on 1,001 AskUbuntu records.
arXiv:2603. 12977v3 Announce Type: replace Abstract: Foundation models are commonly deployed as frozen feature extractors with a small trainable head to adapt to private, user-generated data in federated settings.
The paper investigates the feasibility of forget‑only unlearning, where a deletion algorithm can only access the trained model and the examples to forget, without any retained data or additional training information. It demonstrates that the possibility of successful unlearning depends on the learning method, showing that different datasets can yield identical models yet require distinct post‑deletion outputs. The authors establish lower bounds on the accuracy of unlearning relative to retraining and prove that, for certain learners, the algorithm must memorize a substantial portion of the training data—potentially the entire dataset—to handle arbitrary deletion requests, implying that models designed for forget‑only unlearning may need to retain more information than standard training does.
arXiv:2606. 03808v1 Announce Type: cross Abstract: We propose PURGE, a machine unlearning algorithm built on a simple but an under-exploited observation: continual learning (CL) and machine unlearning (MU) which are fundamentally dual problems.
The paper introduces Executable Release Certification (ExecCert), a layer that certifies machine‑unlearning artifacts at release time. ExecCert either closes a method’s native certificate for the executed candidate or applies Retraining‑Reference Release Verification (RRV) to ensure fidelity to the current retain‑set. An incremental RRV is developed for frozen representations with a mutable ridge head, allowing certified evidence to be maintained across deletion requests without full reconstruction. Experiments on four unlearning implementations show ExecCert preserves valid certificates, tightens bounds, and improves release decisions, while incremental certification remains cost‑effective in sequential‑service settings.
The paper introduces Quantized Sufficient Statistics (QSS), a method that separates a frozen global schema from mutable, sum‑decomposable content to enable exact unlearning. QSS offers two variants: QSS‑L, which removes a label while keeping the input, and QSS‑E, which removes both input and label by learning a schema without deletable examples. Experiments on 15 vision, text, and tabular datasets show that QSS‑L achieves deletion latencies up to 483× faster than SISA while maintaining comparable accuracy, and QSS‑E quantifies the accuracy trade‑off when all traces of an input are removed.
arXiv:2505. 12239v2 Announce Type: replace-cross Abstract: In Continual Learning (CL), using a Pre-Trained Model (PTM) as the feature extractor has become a popular practice.
The paper investigates how federated unlearning systems that broadcast updated linear classifiers after each client update can inadvertently leak the compact, additive summaries used for deletion. By submitting known changes and analyzing the returned classifiers, an attacker can recover the deleted sample’s class or even reinstate it. Experiments on MNIST and CIFAR‑10 show that high‑precision broadcasts enable exact label recovery, while lower precision limits fine‑grained recovery and diverse responses can prevent identification.
The study audits 16 open RLVR math corpora, totaling 1,673,514 training items, against six benchmarks with 1,949 questions to assess the reliability and data‑loss costs of decontamination policies. Eight policies—based on exact, lexical, and semantic matching—were evaluated using 594 dual‑reviewed training‑benchmark pairs, with calibration and held‑out sets ensuring disjoint benchmark questions and training‑example families. Results show that at a cosine threshold of 0.80, 88% of sampled removals were false, and no semantic threshold in the tested grid met a 90% conservative‑precision target; however, the judge‑adjudicated policy achieved 98.1% resolved precision and 98.2% recall on held‑out corpora, with no confirmed false removals among six sampled removals.
Reinforcement learning with verifiable rewards (RLVR) math corpora are routinely screened for benchmark overlap, but the reliability and data-loss costs of cleaning policies remain underexamined. We a...
The paper investigates how machine unlearning for large language models (LLMs) can unintentionally erase related knowledge, even in distant domains. By analyzing the propagation of unlearning effects before any model updates, the authors discover a consistent decay pattern where collateral damage is strongest near the targeted forget set and diminishes with semantic distance but never fully disappears at domain boundaries. They propose a pre-unlearning prediction task—forget-set auditing—to identify potential collateral damage early, finding that interaction features between the forget set and evaluation set are the most predictive signals. This approach offers an early warning system for risky unlearning runs and guides the design of more reliable unlearning procedures.
The paper presents a robust approximate unlearning method for Generalized Learning Vector Quantization (GLVQ) classifiers, enabling the removal of specific data samples from trained models. It evaluates how the size and characteristics of the forget set affect unlearning, proposes an adapted technique to mitigate instabilities, and demonstrates that the method achieves near-zero deviation from a fully retrained model while being up to 10^5 times more efficient on benchmark tabular datasets. The approach is applicable to any GLVQ model without altering its training scheme.
The paper investigates whether a model that has undergone class unlearning can still recover forgotten classes without access to original data. It introduces a white‑box audit method that generates synthetic probes in representation space, filters them by confidence, and relabels boundary‑adjacent probes as the forgotten class. The authors define a Relearning Score to quantify recovery while preserving retain performance, and demonstrate that several unlearning techniques on CIFAR‑10, CIFAR‑100, and TinyImageNet can be fully recovered in a source‑free setting, sometimes even outperforming a retrained reference.