The paper introduces a method for exact unlearning of Gibbs supervised learning algorithms via a variational formulation based on empirical risk minimization with relative entropy regularization (ERM‑RER). By maximizing the expected empirical risk over the data to be removed while regularizing with relative entropy to the original algorithm, the resulting solution is a new Gibbs probability measure that matches the distribution of an algorithm retrained from scratch on the remaining data. The approach also provides a general framework for reweighting data points in ERM‑RER, allowing for up‑ or down‑weighting to control generalization error or other objectives.
By Yaiza Bermudez, Samir M. Perlaza, I\~naki Esnaola
The paper investigates three operations on Gibbs probability measures: renormalization, normalized log-linear combination, and nesting (changing the reference measure). It shows that the measures produced by the second and third operations solve related optimization problems and that, for specific parameters, nesting one Gibbs measure into another is equivalent to log-linearly combining them. This equivalence has practical implications, such as enabling a one-shot federated learning system where clients’ locally trained Gibbs algorithms can be combined on a server to match the performance of a centrally trained Gibbs algorithm.
By Yaiza Bermudez, Samir M. Perlaza, I\~naki Esnaola
arXiv:2608.23960v1 Announce Type: cross
Abstract: Missing labels are usually regarded as a source of information loss in classification. We study a semi-supervised setting in which the probability of...
By You-Gan Wang, Jinran Wu, Geoffrey J. McLachlan
arXiv:2407. 12288v5 Announce Type: replace-cross Abstract: The progress of machine learning over the past decade is undeniable.
By Hong Jun Jeon, Benjamin Van Roy
arXiv:2609.00774v1 Announce Type: cross
Abstract: We consider semi-supervised classification from a partially classified sample arising from a two-component Weibull mixture. The feature is observed f...
By Jinran Wu, You-Gan Wang, Geoffrey J. McLachlan
arXiv:2606. 13984v1 Announce Type: cross Abstract: Decision trees are one of the fundamental tools in statistical learning due to their interpretability, flexibility, and their ability to adapt to nonlinear structures.
By Mathias Bourel