arXiv Statistics ML

Equivalence Between Nested Gibbs Measures and Log-Linear Combinations of Gibbs Measures

The paper investigates three operations on Gibbs probability measures: renormalization, normalized log-linear combination, and nesting (changing the reference measure). It shows that the measures produced by the second and third operations solve related optimization problems and that, for specific parameters, nesting one Gibbs measure into another is equivalent to log-linearly combining them. This equivalence has practical implications, such as enabling a one-shot federated learning system where clients’ locally trained Gibbs algorithms can be combined on a server to match the performance of a centrally trained Gibbs algorithm.

arXiv Machine Learning

Sep 25

Machine Unlearning for Gibbs Supervised Learning Algorithms

The paper introduces a method for exact unlearning of Gibbs supervised learning algorithms via a variational formulation based on empirical risk minimization with relative entropy regularization (ERM‑RER). By maximizing the expected empirical risk over the data to be removed while regularizing with relative entropy to the original algorithm, the resulting solution is a new Gibbs probability measure that matches the distribution of an algorithm retrained from scratch on the remaining data. The approach also provides a general framework for reweighting data points in ERM‑RER, allowing for up‑ or down‑weighting to control generalization error or other objectives.

By Yaiza Bermudez, Samir M. Perlaza, I\~naki Esnaola

safety

arXiv Machine Learning

Jun 30

Data-Driven Energy-Based Learning via Gibbs Measures on Hierarchical Structures

arXiv:2606. 30064v1 Announce Type: new Abstract: We introduce a data-driven probabilistic framework for learning systems based on Gibbs measures on hierarchical structures.

By L. U. Abdullaev, F. Herrera, U. A. Rozikov, M. V. Velasco

arXiv Machine Learning

Jul 7

The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithms

arXiv:2411. 12030v3 Announce Type: replace Abstract: In this paper, the method of gaps, a technique for deriving closed-form expressions in terms of information measures for the generalization error of supervised learning algorithms, is introduced.

By Samir M. Perlaza, Xinying Zou

arXiv Machine Learning

Jul 17

Gibbs randomness-compression proposition

arXiv:2505. 23869v4 Announce Type: replace-cross Abstract: A proposition that connects randomness and compression is put forward via Gibbs entropy over set of measurement vectors associated with a compression process.

By M. S\"uzen

efficiency

arXiv Machine Learning

Jul 7

Gradient Descent as Implicit EM in Distance-Based Neural Models

arXiv:2512. 24780v2 Announce Type: replace Abstract: Neural networks trained with standard objectives exhibit behaviors characteristic of probabilistic inference: soft clustering, prototype specialization, and Bayesian uncertainty tracking.

By Alan Oursland

llms

arXiv Machine Learning

Sep 22

A Distributional Optimisation Perspective on Combining Models in Deep Learning

arXiv:2609.24328v1 Announce Type: new Abstract: Combining predictions from different models can improve performance at machine learning tasks, but the training of the individual models and the rule u...

By Congye Wang, Yan Lin, Zheyang Shen, Matthew A. Fisher, Chris. J. Oates

llms fine-tuning benchmarks

arXiv Machine Learning

4d ago

Variational Mixtures and Multi-Marginal Flow Matching: Advancing Statistical Inference with Biological Applications

arXiv:2609.36911v1 Announce Type: new Abstract: In this thesis I develop methods for statistical inference when the distributions arising from complex biological systems are multi-modal, geometricall...

By Oskar Kviman

diffusion multimodal

arXiv Machine Learning

Jul 28

PYPM-GGD: Pitman-Yor Process Mixture with Generalized Gaussian Density using ADAM

arXiv:2607. 24583v1 Announce Type: new Abstract: Large scale Bayesian nonparametrics (BNP) learner such as Stochastic Variational Inference (SVI) can handle datasets with large class number and large training size at fractional cost.

By Kart-Leong Lim

benchmarks

arXiv Machine Learning

Jun 9

Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency Without Model Sweeps

arXiv:2510. 12744v2 Announce Type: replace-cross Abstract: We develop a unified statistical framework for softmax-gated Gaussian mixture of experts (SGMoE) that addresses three long-standing obstacles in parameter estimation and model selection: (i) non-identifiability of gating parameters up to common translations, (ii) intrinsic gate-expert interactions that induce coupled differential relations in the likelihood, and (iii) the tight numerator-denominator coupling in the softmax-induced conditional density.

By Do Tien Hai, Trung Nguyen Mai, TrungTin Nguyen, Nhat Ho, Binh T. Nguyen, Christopher Drovandi

Hugging Face Trending Papers

Jun 29

Data-Driven Energy-Based Learning via Gibbs Measures on Hierarchical Structures

We introduce a data-driven probabilistic framework for learning systems based on Gibbs measures on hierarchical structures. Unlike standard empirical risk minimization, where a dataset is used to identify a single optimal parameter, our approach transforms the empirical loss function into an interaction potential defining an energy-based model.

arXiv Machine Learning

Jun 18

Smoothness-Based Derandomization of PAC-Bayes Bounds

arXiv:2606. 19105v1 Announce Type: new Abstract: We study PAC-Bayes derandomization for smooth loss functions.

By Alexandre Lemire Paquin, Brahim Chaib-Draa, Philippe Gigu\`ere

arXiv Machine Learning

Jun 3

Mitigating False Credit Propagation: Probabilistic Graphical Reward Aggregation for Rubric-Based Reinforcement Learning

arXiv:2606. 03361v1 Announce Type: new Abstract: Rubric-based rewards are increasingly used for open-ended language model post-training, but criterion-level scores are often aggregated as independent utilities.

By Can Lv, Mingju Chen, Heng Chang, Shiji Zhou

llms reinforcement-learning

Related stories