arXiv Machine Learning

Nonparametric autoregressive estimation of probability distributions with parallel gradient boosting

The paper addresses the challenge of estimating joint probability distributions in high-dimensional settings by factorizing them autoregressively and using nonparametric conditional estimators. It establishes theoretical convergence bounds that link the difficulty of joint estimation to the hardest factor in the factorization, especially when conditional estimators adapt to sparsity in an undirected dependence graph. Empirical results show that an autoregressive estimator built with parallel gradient boosting achieves competitive performance on simulated and real datasets, even without optimizing factorization order.

arXiv Machine Learning
6d ago

Measuring Differences between Conditional Distributions using Kernel Embeddings

The paper introduces a unified framework for measuring divergences between conditional distributions using kernel embeddings, termed conditional maximum mean discrepancy (CMMD). It defines a family of CMMD metrics—CMMD_0, CMMD_1, and CMMD_2—each based on different RKHS embeddings, and generalizes them to a level‑s CMMD with clear assumptions and operator‑based smoothing connections. Additionally, the authors present a novel doubly robust estimator for CMMD and demonstrate through experiments that the metric captures complex conditional dependencies for statistical testing.

By Peter Moskvichev, Siu Lun Chau, Dino Sejdinovic
arXiv Machine Learning
Jun 26

Learning from a Biased Sample

arXiv:2209. 01754v5 Announce Type: replace-cross Abstract: The empirical risk minimization approach to data-driven decision making requires access to training data drawn under the same conditions as those that will be faced when the decision rule is deployed.

By Roshni Sahoo, Lihua Lei, Stefan Wager
arXiv Statistics ML
Aug 25

Model-Agnostic Covariate-Assisted Inference on Partially Identified Causal Effects

The paper introduces a model‑agnostic inference framework for partially identified causal effects that leverages covariate information without requiring discrete covariates or accurate conditional distribution estimates. Using duality theory for optimal transport, the method delivers uniformly valid inference in randomized experiments, is doubly robust in observational settings, achieves asymptotic unbiasedness when nuisance parameters converge semiparametrically, and allows multiplier‑bootstrap selection of covariates and models while remaining computationally efficient. Empirical applications show the approach consistently narrows identified sets and confidence intervals without imposing extra structural assumptions.

By Wenlong Ji, Lihua Lei, Asher Spector
arXiv Machine Learning
Jun 5

Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization

arXiv:2410. 02628v5 Announce Type: replace Abstract: Learning conditional distributions $\pi^*(\cdot|x)$ is a central problem in machine learning, which is typically approached via supervised methods with paired data $(x,y) \sim \pi^*$.

By Mikhail Persiianov, Arip Asadulaev, Nikita Andreev, Nikita Starodubcev, Dmitry Baranchuk, Anastasis Kratsios, Evgeny Burnaev, Alexander Korotin