The paper introduces an amortized learning framework for selecting bandwidths in kernel density estimation by optimizing the logarithmic score across a distribution of tasks. It uses a truncated-and-renormalized bounded-support formulation and affine standardization to achieve stable learning and transferability across different intervals. Experiments on Gaussian samples, a multi-family benchmark, and randomized Gaussian mixtures demonstrate that the learned selector outperforms traditional methods such as Silverman’s rule, Sheather–Jones, and least‑squares cross‑validation, especially for small or heterogeneous samples.
arXiv:2504.05161v2 Announce Type: replace-cross Abstract: Score estimation is the backbone of score-based generative models (SGMs), especially denoising diffusion probabilistic models (DDPMs). A key...
The paper introduces a data‑driven Fourier‑trained neural‑network estimator for fixed‑horizon probability densities, using a positive Gaussian–Laplace mixture with a closed‑form characteristic function. Training occurs directly in Fourier space, enabling non‑negativity and unit‑mass preservation while handling both i.i.d. and resampling‑based pseudo‑sampling data. The authors provide theoretical error bounds, a multidimensional extension, and demonstrate competitive empirical performance against Expectation–Maximization, especially on heavy‑tailed targets.
arXiv:2609.15785v1 Announce Type: cross Abstract: We study density ratio estimation and importance-weighted regression under target shift with continuous outputs. Under target shift, the conditional...
The paper introduces HACK GPs, a method that treats kernel selection for Gaussian Processes as an online learning problem with expert advice. Each candidate kernel is viewed as a GP expert, and a distribution over these experts is updated online using AdaHedge based on a loss that reflects both function fit and task alignment. Two variants—Mixture of Gaussians and categorical sampling—are presented, with theoretical guarantees that the weight concentrates on the best kernel under a loss‑gap condition, and empirical results show robust performance across Bayesian optimization, level set estimation, and Bayesian active learning compared to standard kernels and simple ensembles.
The paper introduces KENDO, a unified framework that combines Ensemble Gaussian Processes with disagreement‑aware acquisition strategies to address hyperparameter selection in Bayesian optimization and active learning. By replacing costly hyperparameter sampling with a kernel ensemble and adaptive Bayesian weighting, KENDO‑BO and KENDO‑AL provide self‑correcting mechanisms tailored to their respective tasks. Experiments on synthetic and real‑world benchmarks show that KENDO‑BO matches or outperforms state‑of‑the‑art methods while cutting computational cost up to fivefold, and KENDO‑AL delivers better predictive calibration with up to 27‑times speedup compared to MCMC‑based baselines.
arXiv:2111. 10722v4 Announce Type: replace-cross Abstract: We propose a novel deterministic sampling method, EVI-MMD, to approximate a target distribution $\rho^*$ by minimizing the kernel discrepancy, also known as the Maximum Mean Discrepancy (MMD).
arXiv:2510. 12744v2 Announce Type: replace-cross Abstract: We develop a unified statistical framework for softmax-gated Gaussian mixture of experts (SGMoE) that addresses three long-standing obstacles in parameter estimation and model selection: (i) non-identifiability of gating parameters up to common translations, (ii) intrinsic gate-expert interactions that induce coupled differential relations in the likelihood, and (iii) the tight numerator-denominator coupling in the softmax-induced conditional density.
arXiv:2607. 19031v1 Announce Type: new Abstract: Automated algorithm selection in black-box optimization typically relies on supervised models that map landscape features to algorithm performance labels.
arXiv:2606. 11469v1 Announce Type: cross Abstract: We study the task of density estimation, where we hope to accurately estimate a probability density from $n$ samples.
The paper introduces a scalable Bayesian method for conditional density estimation using a logistic Gaussian process. It models the log conditional density with a separable covariance— a Matérn kernel on the response dimension and a Nyström-feature kernel on covariates— and samples the latent field via kinetic Langevin dynamics instead of traditional approximations. Experiments on photometric‑redshift data with up to 3.9 million observations show the approach matches state‑of‑the‑art tabular foundation models on density and calibration metrics.
arXiv:2608.21729v1 Announce Type: new Abstract: Simulation-Based Inference (SBI) serves as a vital framework for parameter inference in scientific fields where simulators involve intractable likeliho...
arXiv:2607. 24583v1 Announce Type: new Abstract: Large scale Bayesian nonparametrics (BNP) learner such as Stochastic Variational Inference (SVI) can handle datasets with large class number and large training size at fractional cost.