The paper introduces a pooling‑ridge estimation method for functional linear regression that handles data observed at discrete times, ranging from sparse to dense designs. By combining pooling strategies with RKHS‑based techniques, the authors achieve minimax‑optimal prediction risk for both scalar‑on‑function and function‑on‑function models. The study identifies distinct phase transitions in convergence behavior, with up to three transitions for function‑on‑function regression, and validates the approach through simulations and real data examples.
By Shunxing Yan, Fang Yao
arXiv:2609. 39212v1 Announce Type: new Abstract: We study additive regression under a potentially non-product random design on $[0,1]^d$, allowing the dimension $d$ to grow with the sample size $n$.
By Baptiste Ferrere, Fabrice Gamboa, Jean-Michel Loubes
The paper introduces a learning-based surrogate approach for stochastic optimization problems where uncertainty depends on the decision, modeled via a nonparametric regression. It constructs a surrogate that embeds iteratively updated Jacobian estimates, using an adaptive random design that focuses sampling near the current iterate to achieve dimension‑independent convergence of the Jacobian estimates. The resulting learning‑based stochastic prox‑linear (L‑SPL) algorithm demonstrates nonasymptotic convergence rates and outperforms existing methods in sample efficiency and objective value in numerical experiments.
By Boyang Shen, Junyi Liu
arXiv:2506. 11336v2 Announce Type: replace Abstract: We study the sample complexity of stochastic convex optimization when problem parameters such as the distance to optimality and the Lipschitz constant are unknown.
By Jared Lawrence, Ari Kalinsky, Hannah Bradfield, Yair Carmon, Oliver Hinder
arXiv:2606. 08799v1 Announce Type: cross Abstract: We study the generalization of ridge-regularized nonlinear least-squares models via on-average algorithmic stability, deriving error bounds for local minimizers in terms of a data-dependent effective dimension that reflects the geometry of the gradient model at the trained parameters, through the empirical Jacobian Gram matrix and a residual--curvature term.
By Ayub Kharel, Ilja Kuzborski, Patrick Rebeschini, Yasin Abbasi-Yadkori
arXiv:2609. 03129v1 Announce Type: cross Abstract: Several classical machine-learning methods, such as KRRs and SVRs, are both computationally and analytically tractable since their estimators either admit closed-form expressions or are obtained by minimizing convex training objectives; neither feature is generally available for deep neural networks.
By Ruiyang Hong, Hrad Ghoukasian, Anastasis Kratsios
The paper develops a rigorous framework for computing the Normalized Maximum Likelihood (NML) codelength for regular path‑differentiable Lipschitz (PDL) estimators, which include non‑smooth models such as Lasso and Sparse SVMs. By leveraging geometric measure theory and a novel Propose‑and‑Project Metropolis‑Hastings sampler, the authors provide a method to exactly evaluate the stochastic complexity for these non‑smooth estimators and demonstrate its scalability to high‑dimensional settings. The study shows that the exact NML criterion can match cross‑validation performance while being more data‑efficient, offering a theoretically grounded alternative for model selection in modern machine learning.
By Trenton Lau, Gary P. T. Choi
arXiv:2609.08671v1 Announce Type: cross
Abstract: In this paper, we consider the scalar-on-function linear regression model under a realistic sampling scheme in which the functional covariates are ob...
By Sixtine Sphabmixay
arXiv:2607. 26562v1 Announce Type: cross Abstract: We study optimization under performative prediction, where deploying a model affects the future data distribution.
By Hiroki Hamaguchi, Yuya Hikima, Hiroshi Sawada, Akiko Takeda
arXiv:2411.09686v4 Announce Type: replace
Abstract: Regressing a function $F$ on $\mathbb{R}^d$ without incurring the statistical and computational curse of dimensionality requires exploitable struct...
By Yantao Wu, Mauro Maggioni
arXiv:2609. 38412v1 Announce Type: cross Abstract: We propose a novel local-polynomial estimator of the ratio $r=f/g$ of two $d$-dimensional densities $f$ and $g$, from which independent samples are available.
By Hajo Holzmann, Alexander Meister
arXiv:2609. 25605v1 Announce Type: cross Abstract: In this paper, we study the estimation of a marginal regression function from independent units with repeated binary, count, or continuous responses using ReLU deep neural networks.
By Kexuan Li