← Back to all news
arXiv Statistics ML

Two-sample comparison through additive tree models for density ratios

The paper introduces additive tree models for estimating the density ratio between two distributions, using a novel balancing loss that enables training with supervised learning algorithms such as forward‑stagewise optimization and gradient boosting. The loss also functions as a pseudo‑likelihood, allowing generalized Bayesian inference via backfitting samplers for Bayesian additive regression trees, which provides uncertainty quantification. Experiments show that the boosting algorithms achieve high accuracy with low computational cost, and the Bayesian approach offers competitive accuracy while delivering uncertainty estimates, demonstrated on microbiome compositional data.

Related stories

arXiv Machine Learning
Sep 7

Beyond Homoscedasticity: Decoupled Uncertainty Optimization for Deep Imbalanced Regression

The paper introduces DUO, a framework for Deep Imbalanced Regression that models each prediction as a conditional Gaussian to capture instance‑level uncertainty. By decoupling mean and variance optimization, DUO enhances learning signals for tail samples and mitigates gradient coupling that hampers hard examples. A distribution‑guided contrastive learning component further refines feature representations, leading to state‑of‑the‑art performance on several visual and biological regression benchmarks.

By Juncheng Zhou, Jiaxi Lu, Weijing Zeng, Zhong Li, Hao Qi, Jingsong Cui
benchmarks
More like this →
arXiv Machine Learning
Sep 24

Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate

arXiv:2609. 27930v1 Announce Type: cross Abstract: We propose a Bayesian nonparametric mixture of regression trees with a Dirichlet process prior over tree-parameter pairs, enabling data-driven selection of ensemble size and unifying CART, BART, random forests, and boosting.

By Subhasish Basak, Anik Roy, Sourabh Bhattacharya
efficiencybenchmarkssafety
More like this →
arXiv AI
Oct 1

BayesNDE: Bayesian Generative Modeling for Neural Density Estimation

BayesNDE is a neural density estimator that uses Bayesian generative modeling to estimate densities without relying on invertible networks or Jacobian-determinant calculations. It constructs an adaptive proposal for each observation by inferring a sample-specific latent posterior, and then applies bridge sampling to combine proposal samples with separate posterior samples for density estimation. Experiments on synthetic datasets show improved density estimation and structure recovery, while real-world applications demonstrate better anomaly detection.

By Chenglin Li, Qiao Liu
multimodalbenchmarks
More like this →
arXiv Machine Learning
Aug 12

Hierarchical Empirical-Bayes Naive Bayes: Minimax Smoothing and Calibration with AODE Extension

arXiv:2608. 11162v1 Announce Type: new Abstract: The Naive Bayes (NB) classifier remains a standard choice for categorical data, yet its widely used smoothing rules, such as Laplace, Lidstone, Krichevsky-Trofimov, and the $m$-estimate, all prescribe a fixed smoothing strength that ignores feature cardinality, sample size, and class imbalance, inducing a non-vanishing bias on modern high-cardinality tabular data.

By Nguyen Thai Anh, Truong Viet Vu, Tran Thien Thanh, Vo Nguyen Quoc Bao, Ngo Hoang Tu
benchmarkssafety
More like this →
arXiv Machine Learning
Aug 18

Convolution Smoothed Quantile Regression for XGBoost

arXiv:2608. 15290v1 Announce Type: cross Abstract: The increasing availability of large and complex datasets across many scientific disciplines has led to widespread adoption of machine learning (ML) for prediction.

By Mandy Yao (University of Toronto), Meredith Franklin (University of Toronto)
benchmarks
More like this →
arXiv Machine Learning
Jun 9

Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency Without Model Sweeps

arXiv:2510. 12744v2 Announce Type: replace-cross Abstract: We develop a unified statistical framework for softmax-gated Gaussian mixture of experts (SGMoE) that addresses three long-standing obstacles in parameter estimation and model selection: (i) non-identifiability of gating parameters up to common translations, (ii) intrinsic gate-expert interactions that induce coupled differential relations in the likelihood, and (iii) the tight numerator-denominator coupling in the softmax-induced conditional density.

By Do Tien Hai, Trung Nguyen Mai, TrungTin Nguyen, Nhat Ho, Binh T. Nguyen, Christopher Drovandi
More like this →
arXiv Machine Learning
Oct 12

Product Monte Carlo: Expectation Under a Product of Densities

Product Monte Carlo (PMC) is an algorithm that estimates expectations under a product of densities using only Monte Carlo samples from each individual density. By reformulating the intractable product expectation as a regression problem involving computable conditional expectations, PMC provides both point estimates and coherent Bayesian uncertainty quantification. The method achieves higher accuracy, better-calibrated uncertainty, and improved scalability compared to existing techniques, and it does not require density evaluations, enabling its use in simulation‑based inference such as protein image analysis.

By Pietro Carlotti, Sifan Liu, Yuling Yao
More like this →