arXiv Machine Learning

Conditional-Independence-Regularized Distributional Autoencoders for Mixed-Type Data

The paper introduces Conditional-Independence-Regularized Distributional Autoencoders, a framework for learning low-dimensional representations of mixed-type data that includes both numerical and categorical variables. It uses an energy-score objective for numerical variables, a likelihood objective for categorical variables, and an auxiliary conditional independence regularization term to capture dependencies between variable types. The authors provide theoretical analysis and demonstrate that the method improves categorical distribution recovery, achieves competitive overall conditional distribution recovery, and preserves mixed-type dependence structure on synthetic and real-world datasets.

arXiv Statistics ML
Oct 2

Generalized Engression Models

The paper introduces generalized engression models, a unified nonparametric framework for estimating the full conditional distribution of multivariate outcomes that may include continuous, binary, categorical, ordinal, or ranking variables. Building on the engression deep generative model, it incorporates data‑type‑specific link functions and a stochastic perturbation to smooth the loss, allowing gradient‑based training even with discontinuous links. The authors prove universal representation results for continuous, discrete, and mixed outcomes, and demonstrate that the method matches or surpasses existing type‑specific and joint species distribution models in simulations and two real‑world applications.

By Xinwei Shen, Zijian Guo, Francis Bach
arXiv Machine Learning
Sep 25

Sufficiently Reduced Distributional Regression

Sufficiently Reduced Distributional Regression (SRDR) is a generative approach that merges conditional distribution estimation with nonlinear sufficient dimension reduction (SDR). By framing SDR as a risk minimization problem using strictly proper scoring rules, SRDR jointly learns a dimension reduction map and a generative prediction model through minimization of the energy score, which can be estimated via sampling. The method extends to multi‑environment data and classification, and theoretical results show convergence of estimated conditional distributions in energy distance, implying asymptotic sufficiency. In experiments on CT slice localization, superconductivity data, and digit classification, SRDR recovers low‑dimensional sufficient structure and matches or surpasses state‑of‑the‑art nonlinear SDR methods in representation quality and predictive performance.

By Alexander Henzi, Tiange Liu, Xinwei Shen
arXiv Machine Learning
Oct 9

Marformer: A Transformer for Predicting Missing Data Distributions

The Marformer is a Transformer model designed to predict conditional marginal distributions of missing variables given observed data, enabling computation of Bayes risk and Value of Information. It learns a hidden-vector representation for each variable’s distribution and refines these through attention, similar to BERT’s masked language modeling but for probabilistic inference. Evaluations on synthetic Bayesian networks, discretized Gaussians, structured annotation data, and a real annotation dataset show that the Marformer matches or surpasses classical missing-data methods while being faster and not requiring explicit joint distribution modeling.

By Prabhav Singh, Xiheng Tom Wang, Haojun Shi, Jason Eisner
Hugging Face Trending Papers
Jul 5

Optimal Mixture-of-Experts Model Averaging for Conditional Generative Models

Conditional generative models have emerged as powerful tools for sampling from target conditional distributions, driving substantial advances across a wide range of scientific and applied domains. As these models proliferate, practitioners often face multiple plausible generators whose performance can vary with the task, data, or input condition.