The paper extends the classical conjugate Bayesian framework by leveraging the Fenchel‑Young inequality to define a generalized evidence lower bound that transcends the usual KL geometry. This leads to a new notion of conjugacy—closure of prior‑potential families that remain affine under loss translation—allowing additive canonical updates, parameter‑dependent support, and tunable heavy‑tailed posteriors. The authors also derive a Fenchel‑Young generalization of EM, encompassing both classical and hard EM, and demonstrate its utility in generalized Bayesian linear regression and sparse Gaussian mixture models.
By Sophia Sklaviadis, Thomas Moellenhoff, Emtiyaz Khan, Andre Martins, Mario Figueiredo
arXiv:2603. 09793v2 Announce Type: replace Abstract: Bayesian optimization is a data-efficient technique that has been shown to be extremely powerful to optimize expensive, black-box, and possibly noisy objective functions.
By Federico Pavesi, Antonio Candelieri, No\'emie Jaquier
arXiv:2606. 13984v1 Announce Type: cross Abstract: Decision trees are one of the fundamental tools in statistical learning due to their interpretability, flexibility, and their ability to adapt to nonlinear structures.
By Mathias Bourel
arXiv:2602. 02250v2 Announce Type: replace-cross Abstract: Kullback-Leibler (KL) divergence regularization is widely used in reinforcement learning, but it becomes infinite under support mismatch and can degenerate in low-noise regimes.
By Viktor Stein, Adwait Datar, Nihat Ay
The paper investigates how fast predictive regret guarantees of exact Bayesian online learning can be maintained when using approximate posterior methods. It establishes a general theorem linking the cumulative cost of posterior approximation to the contraction radius of the exact Gibbs posterior and the Wasserstein distance between approximate and exact posteriors. Three concrete online learning scenarios—linear models, infinite‑dimensional exponential families, and Gaussian process regression—illustrate that appropriately accurate approximations (projected Langevin, truncation, and sparse variational posteriors) preserve fast regret bounds while reducing computational demands.
By Ilsang Ohn
arXiv:2512. 01930v2 Announce Type: replace-cross Abstract: Stochastic Variance Reduced Gradient (SVRG) and its variants aim to speed-up training by using gradient corrections.
By Nico Daheim, Thomas M\"ollenhoff, Ming Liang Ang, Mohammad Emtiyaz Khan
The paper surveys recent machine learning and reinforcement learning techniques that enhance black‑box optimization (BBO) solvers, which are used when objective functions can only be queried at a cost and lack gradients. It reviews a range of algorithms—from surrogate‑based methods like mlrMBO and ZO‑AdaMM to transformer‑based optimizers such as B2Opt and diffusion‑model‑based BBO—alongside benchmark efforts such as the NeurIPS 2020 BBO Challenge and MetaBox. A controlled numerical study demonstrates that ML‑based candidate screening can improve evaluation‑budget performance for some direct‑search and differential‑evolution solvers, though the benefit varies by solver.
By Morteza Kimiaei, Vyacheslav Kungurtsev
The paper introduces a contextual bandit algorithm that incorporates concavity constraints into a Bayesian Gaussian Process posterior. By combining a concavity-preserving regression spline basis with a constrained GP, the authors create a tractable shape-constrained estimator and a UCB algorithm. The resulting method achieves regret bounds that are at least as good as standard GP-UCB and can be tighter when concavity information is informative, as shown by experiments on benchmarks and a Warfarin dosing application.
By Kevin Li, Eric Laber
arXiv:2509.10393v5 Announce Type: replace-cross
Abstract: Several methods in statistics and machine learning target a probability distribution for which an entropy-regularised variational objective i...
By Cl\'ementine Chazal, Heishiro Kanagawa, Zheyang Shen, Anna Korba, Chris. J. Oates
The paper studies how a posterior approximation must preserve a specific downstream decision problem. It introduces a regret geometry defined by the target posterior and loss, and uses the forward‑Kullback‑Leibler divergence from a baseline approximation to quantify the information needed for action changes. By analyzing the generalized eigenproblem of the regret Hessian and the baseline information metric, the authors derive a tolerance‑dependent effective dimension and show how restricted approximation families can either miss crucial decision directions or incur excess Fisher information costs.
By Sean Plummer
arXiv:2610.41949v1 Announce Type: cross
Abstract: Few-step flow models generate samples through a handful of large denoising transitions. Applying GRPO to such models requires introducing stochastici...
By Zhuguanyu Wu, Congjia Chen, Ruihao Gong, Yang Yong, Shiqiao Gu, Yi Xu, Xianglong Liu
The paper introduces a new variational inference framework that uses tangent transformations to handle strongly super‑Gaussian likelihoods across a wide range of probability models. By constructing tangent minorants of the log‑likelihood through convex duality, the method achieves conjugacy with Gaussian priors, enabling tractable inference where traditional approaches struggle. The authors provide algorithmic convergence guarantees and near‑parametric risk bounds, and demonstrate superior scalability and accuracy on both simulated and real‑world datasets compared to existing variational algorithms.
By Somjit Roy, Pritam Dey, Debdeep Pati, Bani K. Mallick