← Back to all news
arXiv Machine Learning

Machine Unlearning for Gibbs Supervised Learning Algorithms

The paper introduces a method for exact unlearning of Gibbs supervised learning algorithms via a variational formulation based on empirical risk minimization with relative entropy regularization (ERM‑RER). By maximizing the expected empirical risk over the data to be removed while regularizing with relative entropy to the original algorithm, the resulting solution is a new Gibbs probability measure that matches the distribution of an algorithm retrained from scratch on the remaining data. The approach also provides a general framework for reweighting data points in ERM‑RER, allowing for up‑ or down‑weighting to control generalization error or other objectives.

Related stories

arXiv Statistics ML
Sep 18

Equivalence Between Nested Gibbs Measures and Log-Linear Combinations of Gibbs Measures

The paper investigates three operations on Gibbs probability measures: renormalization, normalized log-linear combination, and nesting (changing the reference measure). It shows that the measures produced by the second and third operations solve related optimization problems and that, for specific parameters, nesting one Gibbs measure into another is equivalent to log-linearly combining them. This equivalence has practical implications, such as enabling a one-shot federated learning system where clients’ locally trained Gibbs algorithms can be combined on a server to match the performance of a centrally trained Gibbs algorithm.

By Yaiza Bermudez, Samir M. Perlaza, I\~naki Esnaola
More like this →
arXiv AI
Jul 7

Machine Unlearning via Information Theoretic Regularization

arXiv:2502. 05684v5 Announce Type: replace-cross Abstract: How can we effectively remove or ``unlearn'' undesirable information, such as specific features or the influence of individual data points, from a learning outcome while minimizing utility loss and ensuring rigorous guarantees?

By Shizhou Xu, Thomas Strohmer
More like this →
arXiv Machine Learning
4d ago

Reference-Guided Machine Unlearning

Reference-Guided Machine Unlearning (ReGUn) is a vision unlearning framework that prioritizes distributional indistinguishability over degradation-based heuristics. It uses disjoint held-out data to create a class-conditioned reference distribution for distillation, guiding forget samples toward non-member behavior without explicitly degrading predictions. Experiments across various architectures and datasets show that ReGUn achieves a competitive forgetting–utility trade-off and closely matches retrain-like membership inference behavior.

By Jonas Mirlach, Sonia Laguna, Julia E. Vogt
efficiency
More like this →
arXiv Machine Learning
Jun 26

Learning from a Biased Sample

arXiv:2209. 01754v5 Announce Type: replace-cross Abstract: The empirical risk minimization approach to data-driven decision making requires access to training data drawn under the same conditions as those that will be faced when the decision rule is deployed.

By Roshni Sahoo, Lihua Lei, Stefan Wager
safety
More like this →