The paper develops a rigorous framework for computing the Normalized Maximum Likelihood (NML) codelength for regular path‑differentiable Lipschitz (PDL) estimators, which include non‑smooth models such as Lasso and Sparse SVMs. By leveraging geometric measure theory and a novel Propose‑and‑Project Metropolis‑Hastings sampler, the authors provide a method to exactly evaluate the stochastic complexity for these non‑smooth estimators and demonstrate its scalability to high‑dimensional settings. The study shows that the exact NML criterion can match cross‑validation performance while being more data‑efficient, offering a theoretically grounded alternative for model selection in modern machine learning.
By Trenton Lau, Gary P. T. Choi
arXiv:2607. 07840v1 Announce Type: cross Abstract: Gradient estimation -- the task of computing the gradient of the expected value of a probabilistic program -- has diverse applications in scientific computing, but is notoriously difficult because of issues such as high-dimensional integration, discrete random choices, and complex stochastic dependencies.
By Gaurav Arya, Mathieu Huot, Moritz Schauer, Alexander K. Lew, Feras A. Saad
arXiv:2608. 12009v1 Announce Type: cross Abstract: Bregman proximal stochastic gradient (BPSG) methods bring variance-reduced composite optimization to objectives whose geometry is poorly captured by Euclidean smoothness.
By Chenhan Jin, Shengze Xu, Binghui Xie, Kaiwen Zhou, Fan Jia, James Cheng, Tieyong Zeng
The paper introduces Loopless Variance-Reduced Policy Gradient (LVR-PG), a framework that unifies first‑order estimators combining recursive updates with defensive importance sampling, such as PAGE-PG and STORM-PG. It proves that LVR-PG achieves a ∫O(ε⁻²) last‑iterate global convergence rate under weak gradient domination for general parametric policies, using only first‑order updates and minimal assumptions. By extending lower bounds to continuous parametric policies, the authors show an Ω(ε⁻²) sample complexity limit, establishing that LVR-PG is minimax‑optimal up to logarithmic factors and horizon terms.
By Leonardo Cesani, Matteo Papini, Marcello Restelli
The paper investigates whether momentum-based acceleration can improve Gaussian variational inference (GVI) for strongly log-concave, log-smooth targets. With exact Gaussian expectations, GVI achieves the classic accelerated √κ convergence rate, but when only pointwise derivatives are available, the authors construct targets that force any randomized algorithm to use κ^{1-o(1)} queries, showing curvature alone does not guarantee acceleration. They identify additional structure—specifically a Hessian-Lipschitz condition with small Hessian variation—under which variance-reduced stochastic methods can achieve accelerated guarantees, and demonstrate empirical speedups on real datasets.
By Jaesung R. Park, Uijeong Jang, Yifan Chen
arXiv:2607. 03871v1 Announce Type: new Abstract: Minimum maximum mean discrepancy (MMD) estimation has emerged as a robust and likelihood-free alternative to maximum likelihood estimation for parameter estimation.
By Sophia Seulkee Kang, Louis Sharrock, Xiaoyuan Cheng, Fran\c{c}ois-Xavier Briol, Zonghao Chen
arXiv:2511.03087v3 Announce Type: replace-cross
Abstract: Generalized linear models (GLMs) are fundamental tools for statistical modeling, with maximum likelihood estimation (MLE) serving as the clas...
By Linglingzhi Zhu, Jonghyeok Lee, Yao Xie
arXiv:2610.22763v1 Announce Type: new
Abstract: Bayesian optimization (BO) over discrete spaces is challenging due to missing gradients and the sparsity of valid solutions. Latent-space BO sidesteps...
By Tommaso Ancilli, Thomas Wedenig, Henry Moss, Robert Peharz
arXiv:2411.09686v4 Announce Type: replace
Abstract: Regressing a function $F$ on $\mathbb{R}^d$ without incurring the statistical and computational curse of dimensionality requires exploitable struct...
By Yantao Wu, Mauro Maggioni
The paper introduces a new variational inference framework that uses tangent transformations to handle strongly super‑Gaussian likelihoods across a wide range of probability models. By constructing tangent minorants of the log‑likelihood through convex duality, the method achieves conjugacy with Gaussian priors, enabling tractable inference where traditional approaches struggle. The authors provide algorithmic convergence guarantees and near‑parametric risk bounds, and demonstrate superior scalability and accuracy on both simulated and real‑world datasets compared to existing variational algorithms.
By Somjit Roy, Pritam Dey, Debdeep Pati, Bani K. Mallick
arXiv:2501. 12982v3 Announce Type: replace-cross Abstract: This paper investigates how diffusion generative models leverage (unknown) low-dimensional structure to accelerate sampling.
By Jiadong Liang, Zhihan Huang, Yuxin Chen
arXiv:2610. 02716v1 Announce Type: new Abstract: Objective perturbation adds a random linear term to a regularized empirical risk and releases the exact perturbed minimizer.
By Austin Watkins, Raman Arora