arXiv Machine Learning

Scalable Learning of Determinantal Point Processes with Kernel-Structured Interactions

The paper introduces a scalable approach to learning Determinantal Point Processes (DPPs) by modeling full‑rank interactions through a nonlinear kernel function and latent features. It proposes a surrogate function for maximum likelihood estimation and employs stochastic optimization with random features of the nonlinear kernel. Experiments on real‑world datasets show that this method improves the trade‑off between expressive power and computational scalability, achieving better generalization with fewer parameters.

arXiv Machine Learning
Oct 12

Generalized Determinantal Point Processes for Attraction and Repulsion with Distinct Interaction Kernels

The paper introduces the determinant-ratio point process (DRPP), a new model that captures both attraction and repulsion in subset selection by using a ratio of determinants from two symmetric positive-definite kernels. DRPP retains the feature-based interpretability of standard determinantal point processes while allowing separate specification of attraction and repulsion effects. The authors provide an auxiliary-variable representation that enables efficient MCMC sampling and tractable expectation formulas, facilitating maximum-likelihood learning, and theoretically show that DRPP can realize clustered attraction with fewer parameters than nonsymmetric DPPs.

By Hideaki Ishibashi, Takahiro Kawashima, Hideitsu Hino
arXiv Machine Learning
Sep 17

A General Kernel Framework for Non-CND Distance Measures Using |D|-Dimensional Sparse Landmark Embeddings

The paper introduces the Sparse Landmark Embedding (SLE) kernel, a new framework that removes the need for conditionally negative definite (CND) distance measures in kernel methods and Gaussian Processes. By embedding each input into a sparse feature vector using compactly supported bump functions centered at all training points, any standard positive semi-definite (PSD) kernel can be applied in this embedding space, guaranteeing PSD for arbitrary distance measures. The authors provide theoretical guarantees on PSD, sparsity, stability, and universal approximation, and show through experiments with geodesic and Wasserstein distances that the SLE kernel matches or surpasses domain-specific baselines in predictive accuracy and uncertainty quantification.

By Marcus M. Noack, Maher B. Alghalayini, Mark D. Risser
arXiv Machine Learning
Sep 16

A Weighted Kernel Method for Approximation that Adapts to Learned Multivariable Structure

The paper introduces Total Sensitivity Kernels (TSKs), a weighted ANOVA kernel framework that learns the importance of individual inputs and their interactions for approximating a multivariable black-box function from limited data. By selecting an RKHS where the target function has minimum norm, the authors derive a unique solution and prove consistency for finite-data interpolation. Numerical experiments show that adapting the kernel to the learned multivariable structure can significantly improve approximation accuracy compared to a standard product kernel.

By John E. Darges, Laura Weidensager
arXiv Machine Learning
Oct 5

Kernel Singular Value Decomposition with Extension to Multiple Data Sources

Kernel Singular Value Decomposition (KSVD) learns singular vectors for an asymmetric kernel matrix derived from two data sources. The paper extends KSVD to multiple data sources, introducing eKSVD, which jointly learns nonlinear feature mappings and captures maximal information across all sources while incorporating pair‑wise couplings. The authors derive a dual optimization framework, a covariance‑based approach, and demonstrate the effectiveness of eKSVD and neural‑network‑based feature mappings through numerical experiments.

By Xinjie Zeng, Qinghua Tao, Johan Suykens
arXiv Machine Learning
Oct 9

Feature Space Adaptation for Effortless Gaussian Process Flows

The paper introduces improvements to FlowGP, a method for conditional sampling from Gaussian processes outside the linear-Gaussian regime. It adds kernel approximations to scale to high-resolution domains and a new technique for computing marginal likelihood by measuring the work required to steer diffusion toward conditioning statements. These advances enable hyperparameter optimisation within FlowGP and are demonstrated on tasks such as probabilistic downscaling, PDE solution inference on irregular domains, and sea level anomaly field recovery from non-Gaussian satellite data.

By Thomas Cowperthwaite, Louis Sharrock, Lachlan Astfalck, Henry Moss
arXiv Machine Learning
Aug 13

A Variational Analysis of Kernel Learning with Learnable Linear Transformations

arXiv:2502. 11665v3 Announce Type: replace-cross Abstract: The classical kernel ridge regression problem aims to find the best fit for the output $Y$ as a function of the input data $X\in \mathbb{R}^d$, with a fixed choice of regularization term imposed by a given choice of a reproducing kernel Hilbert space, such as a Sobolev space.

By Yang Li, Feng Ruan