← Back to all news
arXiv Machine Learning By Sophia Sklaviadis, Thomas Moellenhoff, Emtiyaz Khan, Andre Martins, Mario Figueiredo

The Fenchel-Young Generalization of Bayesian Principles

Read the original on arXiv Machine Learning →

The paper extends the classical conjugate Bayesian framework by leveraging the Fenchel‑Young inequality to define a generalized evidence lower bound that transcends the usual KL geometry. This leads to a new notion of conjugacy—closure of prior‑potential families that remain affine under loss translation—allowing additive canonical updates, parameter‑dependent support, and tunable heavy‑tailed posteriors. The authors also derive a Fenchel‑Young generalization of EM, encompassing both classical and hard EM, and demonstrate its utility in generalized Bayesian linear regression and sparse Gaussian mixture models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Oct 23

The Bayesian Learning Rule Beyond KL Geometry

The paper investigates how the Bayesian Learning Rule (BLR), traditionally tied to Kullback‑Leibler divergence, can be extended to other geometries. It demonstrates that additive BLR updates are possible using mixed‑space Bregman or Fenchel‑Young discrepancies, leading to generalized maximum‑entropy families and new canonical gradient updates. For example, applying the Tsallis negentropy with quadratic statistics yields bounded‑support, heavy‑tailed q‑Gaussian posteriors and a simple Newton‑like implementation, with experiments on CIFAR‑100 showing controllable support and tail behavior and competitive performance.

By Sophia Sklaviadis, Wu Lin, Andre Martins, Mario Figueiredo, Thomas Moellenhoff, Emtiyaz Khan
More like this →
arXiv Machine Learning
Sep 14

A Generalized Tangent Approximation based Variational Inference Framework for Strongly Super-Gaussian Likelihoods

The paper introduces a new variational inference framework that uses tangent transformations to handle strongly super‑Gaussian likelihoods across a wide range of probability models. By constructing tangent minorants of the log‑likelihood through convex duality, the method achieves conjugacy with Gaussian priors, enabling tractable inference where traditional approaches struggle. The authors provide algorithmic convergence guarantees and near‑parametric risk bounds, and demonstrate superior scalability and accuracy on both simulated and real‑world datasets compared to existing variational algorithms.

By Somjit Roy, Pritam Dey, Debdeep Pati, Bani K. Mallick
benchmarks
More like this →
arXiv Machine Learning
Oct 12

Two-way Guidance: Simultaneous Learning of Summary Statistics for Generative Bayes

The paper introduces a two-way guidance framework that jointly learns summary statistics and posterior distributions for Bayesian simulation-based inference. By imposing a structural restriction on the learned summary, the method ensures that the posterior remains sufficient, while the posterior objective simultaneously steers the summary toward sufficiency. The authors provide theoretical guarantees on posterior estimation error and the deviation of the learned summary from sufficiency, outlining conditions on simulation budgets needed to recover increasingly concentrated posteriors as observation size grows.

By Percy S. Zhai, Max Pachman, Peverel D. Shipley, Jungeum Kim
More like this →