← Back to all news
arXiv Machine Learning By Sophia Sklaviadis, Wu Lin, Andre Martins, Mario Figueiredo, Thomas Moellenhoff, Emtiyaz Khan

The Bayesian Learning Rule Beyond KL Geometry

Read the original on arXiv Machine Learning →

The paper investigates how the Bayesian Learning Rule (BLR), traditionally tied to Kullback‑Leibler divergence, can be extended to other geometries. It demonstrates that additive BLR updates are possible using mixed‑space Bregman or Fenchel‑Young discrepancies, leading to generalized maximum‑entropy families and new canonical gradient updates. For example, applying the Tsallis negentropy with quadratic statistics yields bounded‑support, heavy‑tailed q‑Gaussian posteriors and a simple Newton‑like implementation, with experiments on CIFAR‑100 showing controllable support and tail behavior and competitive performance.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 23

The Fenchel-Young Generalization of Bayesian Principles

The paper extends the classical conjugate Bayesian framework by leveraging the Fenchel‑Young inequality to define a generalized evidence lower bound that transcends the usual KL geometry. This leads to a new notion of conjugacy—closure of prior‑potential families that remain affine under loss translation—allowing additive canonical updates, parameter‑dependent support, and tunable heavy‑tailed posteriors. The authors also derive a Fenchel‑Young generalization of EM, encompassing both classical and hard EM, and demonstrate its utility in generalized Bayesian linear regression and sparse Gaussian mixture models.

By Sophia Sklaviadis, Thomas Moellenhoff, Emtiyaz Khan, Andre Martins, Mario Figueiredo
More like this →
arXiv Machine Learning
Aug 27

Fast rates in Bayesian online learning with approximate posteriors

The paper investigates how fast predictive regret guarantees of exact Bayesian online learning can be maintained when using approximate posterior methods. It establishes a general theorem linking the cumulative cost of posterior approximation to the contraction radius of the exact Gibbs posterior and the Wasserstein distance between approximate and exact posteriors. Three concrete online learning scenarios—linear models, infinite‑dimensional exponential families, and Gaussian process regression—illustrate that appropriately accurate approximations (projected Langevin, truncation, and sparse variational posteriors) preserve fast regret bounds while reducing computational demands.

By Ilsang Ohn
More like this →
arXiv AI
Jun 9

SVRG and Beyond via Posterior Correction

arXiv:2512. 01930v2 Announce Type: replace-cross Abstract: Stochastic Variance Reduced Gradient (SVRG) and its variants aim to speed-up training by using gradient corrections.

By Nico Daheim, Thomas M\"ollenhoff, Ming Liang Ang, Mohammad Emtiyaz Khan
More like this →