arXiv Statistics ML

Scalable and Calibrated Sampling for Bayesian Generalized Linear Mixed Model via Stochastic Gradient Markov Chain Monte Carlo

The paper introduces a stochastic gradient Markov Chain Monte Carlo (SGMCMC) algorithm designed for Bayesian generalized linear mixed models (GLMMs). By employing Fisher's identity, it constructs a biased Monte Carlo estimator of the gradient of the marginal log-likelihood, enabling scalable inference even when direct gradient computation is infeasible. The authors provide a post-hoc covariance correction to calibrate posterior uncertainty and demonstrate the method’s accuracy and practical impact on large-scale biomedical data, notably electronic health records.

arXiv Machine Learning
Sep 14

A Generalized Tangent Approximation based Variational Inference Framework for Strongly Super-Gaussian Likelihoods

The paper introduces a new variational inference framework that uses tangent transformations to handle strongly super‑Gaussian likelihoods across a wide range of probability models. By constructing tangent minorants of the log‑likelihood through convex duality, the method achieves conjugacy with Gaussian priors, enabling tractable inference where traditional approaches struggle. The authors provide algorithmic convergence guarantees and near‑parametric risk bounds, and demonstrate superior scalability and accuracy on both simulated and real‑world datasets compared to existing variational algorithms.

By Somjit Roy, Pritam Dey, Debdeep Pati, Bani K. Mallick
arXiv Machine Learning
Jul 7

Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data

arXiv:2607. 04647v1 Announce Type: cross Abstract: Scalable Bayesian inference for generalized linear mixed models (GLMMs) provides uncertainty-aware analysis of correlated longitudinal data, but existing scalable approaches largely assume low-dimensional tabular predictors and do not directly accommodate high-dimensional modalities such as images and text.

By Yuankang Zhao, Youngsoo Baek, Felipe A. Medeiros, Samuel Berchuck, Matthew M. Engelhard
arXiv Machine Learning
Jun 19

Variational Consensus Monte Carlo for Bayesian Mixture

arXiv:2606. 19643v1 Announce Type: cross Abstract: Motivated by the privacy, sensitivity and sharing limitations of health data, we present a comprehensive pipeline for inference of Bayesian mixture models within a federated learning setting, i.

By Julie Fendler, Francesca L. Crowe, Tom Marshall, Sylvia Richardson, Paul D. W. Kirk
arXiv Machine Learning
Oct 12

Product Monte Carlo: Expectation Under a Product of Densities

Product Monte Carlo (PMC) is an algorithm that estimates expectations under a product of densities using only Monte Carlo samples from each individual density. By reformulating the intractable product expectation as a regression problem involving computable conditional expectations, PMC provides both point estimates and coherent Bayesian uncertainty quantification. The method achieves higher accuracy, better-calibrated uncertainty, and improved scalability compared to existing techniques, and it does not require density evaluations, enabling its use in simulation‑based inference such as protein image analysis.

By Pietro Carlotti, Sifan Liu, Yuling Yao
arXiv Statistics ML
Nov 9

Calibrating your confidence requires thinking of all the ways you could be wrong

The paper introduces a novel Bayesian posterior—a degenerate Gaussian with covariance aligned to the null space of the Fisher information matrix—to address deep learning’s confidence miscalibration. Two efficient algorithms enable scalable posterior sampling and direct predictive confidence computation, achieving state‑of‑the‑art calibration and out‑of‑distribution detection on large models. The work also provides theoretical insights via information geometry, corrects Laplace approximation misalignments, and connects to PAC‑Bayes generalization bounds.

By Marco Miani
arXiv Machine Learning
Jun 2

Accurate Large-sample Uncertainty Quantification using Stochastic Gradient Markov Chain Monte Carlo

arXiv:2606. 00293v1 Announce Type: new Abstract: Tuning algorithms such as stochastic gradient descent (SGD) and stochastic gradient Langevin dynamics (SGLD) for approximate sampling and uncertainty quantification remains challenging, particularly in the practically relevant settings when the batch size is large or the model is misspecified.

By Yu Wang, Jie Ding, Jonathan H. Huggins
arXiv Machine Learning
Aug 28

A Flexible Empirical Bayes Approach to Generalized Linear Models, with Applications to Sparse Logistic Regression

The paper presents a tuning‑free empirical Bayes framework for Bayesian generalized linear models that uses a novel mean‑field variational inference algorithm. By estimating the prior within the VI procedure and optimizing the posterior mean directly, the method reduces optimization complexity and supports scalable solvers like L‑BFGS and stochastic gradient descent. Applied to sparse logistic regression, the approach shows superior predictive performance compared to existing methods.

By Dongyue Xie, Matthew Stephens
arXiv Statistics ML
Aug 28

A Two-step Metropolis Hastings Method for Bayesian Empirical Likelihood Computation with Application to Quantile Regression and Bayesian Model Selection

The paper introduces a two-step Metropolis–Hastings algorithm designed to efficiently sample from Bayesian empirical likelihood (BayesEL) posterior distributions, addressing challenges posed by the complex, often non‑convex support of empirical likelihood. The method leverages current parameter values and estimating equations to propose new values for remaining parameters, making it suitable for problems with discontinuous estimating equations such as simultaneous quantile regression. Additionally, the approach extends naturally to BayesEL model selection via reversible‑jump MCMC, and the authors demonstrate its utility through several real‑life applications.

By Sanjay Chaudhuri, Teng Yin, Snehashis Chakraborty, Rupsa Roy