arXiv Machine Learning

Counterexamples and Sufficient Conditions: Comments on "Optimally-Transported Generalized Method of Moments"

The paper critiques the optimally‑transported generalized method of moments (OTGMM) estimator introduced by Schennach & Starck (2026a), presenting counterexamples that invalidate Theorems 2–6 under their stated assumptions. It shows that the small‑error assumptions fail to guarantee consistency and asymptotic normality, and that the large‑error analysis leads to a modified‑moment GMM estimator that can diverge from the true OTGMM minimizer, even in simple scalar and overidentified models. The authors further provide a sufficient condition ensuring that solutions of the modified moment equations also solve the original constrained problem, and replace a problematic Assumption 16 with a matrix condition that restores the required bound.

arXiv Machine Learning
6d ago

On the SoS Certifiability of Log-Concave Distributions

arXiv:2609. 30105v1 Announce Type: new Abstract: For an arbitrary isotropic log-concave distribution $P$ on $\mathbb{R}^d$, we prove that the polynomial $(Cm)^m\|v\|_2^m - \mathbb{E}_{X\sim P}\langle X,v\rangle^m$ is a sum of squares for every even $m\ge2$, where $C>0$ is a universal constant.

By Aleksandr Storozhenko
arXiv Machine Learning
Jul 27

On the Convergence of Stochastic Low-Rank Adaptation

arXiv:2607. 21975v1 Announce Type: new Abstract: Low-rank adaptation (LoRA) optimizes $J(B,A)=\mathcal L(W_\mathrm{base}+sBA)$ over two adapters $B \in \mathbb{R}^{m \times r}$ and $A \in \mathbb{R}^{r \times n}$ that form a low-rank update to a frozen pretrained weight matrix $W_\mathrm{base} \in \mathbb{R}^{m \times n}$.

By Ru Wang, Chengchang Liu, John C. S. Lui
arXiv Statistics ML
Sep 7

On the Asymptotic Inadmissibility of Double Machine Learning Estimators Under Structure-Agnostic Models

The paper investigates Double Machine Learning (DML) estimators under structure‑agnostic (SA) models, which assume the data‑generating law lies within a neighborhood of fixed machine‑learning estimates. It shows that for two of three studied functionals—the quadratic functional in the Gaussian sequence model and the quadratic density integral functional—the DML estimators are asymptotically inadmissible, being dominated by second‑order empirical higher‑order influence function (HOIF) estimators. For the third functional, the expected conditional covariance, both DML and HOIF estimators remain minimax but neither dominates the other.

By Lin Liu, Rajarshi Mukherjee, James M Robins
Hugging Face Trending Papers
Sep 24

On the SoS Certifiability of Log-Concave Distributions

For an arbitrary isotropic log-concave distribution $P$ on $\mathbb{R}^d$, we prove that the polynomial $(Cm)^m\|v\|_2^m - \mathbb{E}_{X\sim P}\langle X,v\rangle^m$ is a sum of squares for every even $m\ge2$, where $C>0$ is a universal constant. This removes the dependence on the Poincaré constant in the theorem of Kothari and Steinhardt (arXiv:1711.

arXiv Machine Learning
Jul 14

Demixing Sparse Signals from Nonlinear Observations using Generalized Non-convex Regularization

arXiv:2607. 10618v1 Announce Type: cross Abstract: We consider the recovery of a pair of sparse vectors from a limited number of nonlinear observations of their superposition: $y_i=g(\inner{\ba_i}{\bPhi\bw^\ast+\bPsi\bz^\ast})+e_i$, $i=1,\dots,m$, with $m\ll n$, incoherent orthonormal bases $\bPhi,\bPsi$, a scalar link $g$, and noise $e_i$ that may be heavy-tailed or contaminated.

By Raziyeh Takbiri
arXiv Machine Learning
Sep 15

Riemannian ascent--descent for nonconvex nonconcave minimax landscapes: convergence to basin saddle points and applications to distributionally robust optimization

The paper introduces a new convergence framework for solving distributionally robust optimization problems formulated as nonconvex, nonconcave minimax problems over a Euclidean space and a Riemannian manifold. It defines a "basin saddle point"—a locally defined Nash equilibrium—and proves that a Riemannian gradient ascent–descent algorithm converges to such points under a local Łojasiewicz growth condition. The authors apply this theory to a statistical risk DRO problem over Gaussian measures, deriving explicit convergence rates and constants in terms of data dimension, loss moments, and reference covariance.

By Rishabh Dixit, Pranav Upadrashta, Alex Cloninger