← Back to all news
arXiv Machine Learning

Approximation of differential entropy in Bayesian optimal experimental design

The paper presents a method for approximating differential entropy in Bayesian optimal experimental design, focusing on cases where the likelihood entropy is design‑independent or explicitly computable, such as additive noise models. By reducing the expected information gain to a single integral, the authors propose using Monte Carlo or quasi‑Monte Carlo surrogates for the evidence density while evaluating differential entropy with standard techniques, thereby avoiding extra likelihood evaluations. Theoretical analysis shows convergence rates comparable to or better than existing methods, and numerical experiments support these findings.

Related stories

arXiv Machine Learning
Oct 19

Robust Experimental Design via Generalised Bayesian Inference

The paper introduces Generalised Bayesian Optimal Experimental Design (GBOED), extending generalised Bayesian inference to experimental design. It replaces the likelihood in Bayesian updates with a loss function, deriving a new acquisition function called Gibbs expected information gain. Empirical results show that GBOED improves robustness against outliers and mis-specified outcome noise distributions.

By Yasir Zubayr Barlas, Sabina J. Sloman, Samuel Kaski
More like this →
arXiv Machine Learning
Oct 9

Exploiting Gradients in Bayesian Inference of Expensive Simulators

The paper explores how to use gradients of simulator outputs with respect to input parameters to improve Bayesian inference for expensive simulators. By incorporating gradient information into Gaussian process surrogate models, the authors show that Bayesian optimization can converge faster under a limited simulation budget. They find that reverse‑mode differentiation yields efficiency gains, while forward‑mode does not offset its computational cost, especially when the number of parameters exceeds the output dimensionality.

By \v{S}imon Sold\'at, V\'aclav \v{S}m\'idl
efficiency
More like this →
arXiv Machine Learning
Oct 5

Flow Matching for Fast Posterior Sampling in Bayesian Inverse Problems

The paper introduces flow matching as an amortized alternative to Markov chain Monte Carlo for sampling from posterior distributions in Bayesian inverse problems. By training a transport map on joint samples of parameters and data, the method produces independent approximate posterior samples for any observation at negligible online cost, eliminating the need for new likelihood evaluations. The authors provide accuracy estimates in total‑variation distance and Kullback‑Leibler divergence, propose a hybrid Metropolized sampler for exactness, and validate the approach on examples such as electrical impedance tomography and a likelihood‑free Lotka‑Volterra model.

By Jan Blechschmidt, Oliver G. Ernst, Moritz Poguntke, Bj\"orn Sprungk
diffusion
More like this →
arXiv Machine Learning
Sep 14

A Generalized Tangent Approximation based Variational Inference Framework for Strongly Super-Gaussian Likelihoods

The paper introduces a new variational inference framework that uses tangent transformations to handle strongly super‑Gaussian likelihoods across a wide range of probability models. By constructing tangent minorants of the log‑likelihood through convex duality, the method achieves conjugacy with Gaussian priors, enabling tractable inference where traditional approaches struggle. The authors provide algorithmic convergence guarantees and near‑parametric risk bounds, and demonstrate superior scalability and accuracy on both simulated and real‑world datasets compared to existing variational algorithms.

By Somjit Roy, Pritam Dey, Debdeep Pati, Bani K. Mallick
benchmarks
More like this →