arXiv Machine Learning
Jul 24
Zero-Flow Two-Sample Tests
arXiv:2607. 21542v1 Announce Type: new Abstract: We propose a new approach to two-sample testing for deciding whether two sets of samples are drawn from the same distribution.
A nonparametric two-sample test using a parametric integral probability metric
arXiv:2606. 16941v1 Announce Type: cross Abstract: Detecting distributional differences between two independent samples is a fundamental problem in statistics and machine learning.
Related stories
arXiv Machine Learning
Sep 11
Statistically Valid Post-Training Hyperparameter Selection: From Tuning to Guarantees
The paper introduces a statistical framework for post‑training hyperparameter selection, emphasizing the learn‑then‑test (LTT) paradigm. It treats hyperparameter tuning as a multiple hypothesis testing problem over a candidate set, enabling the selection of hyperparameters that meet specified reliability constraints such as risk bounds or information‑theoretic limits. The framework provides finite‑sample control of error probabilities using p‑values, e‑values, and concentration inequalities derived from first principles.
arXiv Machine Learning
Jun 2
A Theoretical Framework for Statistical Evaluability of Generative Models
arXiv:2604. 05324v2 Announce Type: replace Abstract: Statistical evaluation aims to estimate the generalization performance of a model using held-out i.
arXiv Machine Learning
Jun 9
LOTTERY: Learning from Reference-Only Samples in Two-Sample Testing under Size Asymmetry
arXiv:2606. 08460v1 Announce Type: cross Abstract: Data-adaptive two-sample testing assesses if two samples come from the same distribution, using a discrepancy learned from the data (e.
arXiv Machine Learning
Aug 4
Nonparametric Distribution Regression Re-calibration
arXiv:2602. 13362v2 Announce Type: replace-cross Abstract: A key challenge in probabilistic regression is ensuring that predictive distributions accurately reflect true empirical uncertainty.
arXiv Machine Learning
Jul 24
Statistical Inference for Generative Model Comparison
arXiv:2501. 18897v4 Announce Type: replace-cross Abstract: Generative models have achieved remarkable success across a range of applications, yet their evaluation still lacks principled uncertainty quantification.
arXiv AI
Jul 13
Machine Learning for Network Attacks Classification and Statistical Evaluation of Adversarial Learning Methodologies for Synthetic Data Generation
arXiv:2603. 17717v4 Announce Type: replace-cross Abstract: Supervised detection of network attacks has always been a critical part of network intrusion detection systems (NIDS).
arXiv AI
Aug 18
DirMixE: Harnessing Test Agnostic Long-tail Recognition with Hierarchical Label Variations
arXiv:2405. 07780v3 Announce Type: replace-cross Abstract: This paper explores test-agnostic long-tail recognition, a challenging long-tail task where the test label distributions are unknown and arbitrarily imbalanced.
arXiv Machine Learning
Jun 9
Are Two Datasets Close Enough With Statistical Significance? A Kernel Distributional Closeness Testing Approach
arXiv:2507. 12843v3 Announce Type: replace Abstract: Are two distributions close to each other with statistical significance?
arXiv Machine Learning
Jul 23
Directional Kernel Mean Difference: A Fast Signed Statistic for Univariate Distribution Comparison
arXiv:2607. 20119v1 Announce Type: cross Abstract: We introduce the Directional Kernel Mean Difference (DKMD), a signed statistic for univariate distribution comparison that preserves the direction of distributional shifts.
arXiv Machine Learning
Jul 10
Multi-Distribution Robust Conformal Prediction
arXiv:2601. 02998v2 Announce Type: replace Abstract: In many fairness and distribution robustness problems, one has access to labeled data from multiple source distributions yet the test data may come from an arbitrary member or a mixture of them.
arXiv Statistics ML
3d ago
PTED: A multi-dimensional two-sample test for scientific inference and generative machine learning
The article introduces PTED, a Python implementation of a permutation test based on the Energy Distance for two-sample testing in multiple dimensions. PTED uses pairwise distances to compute a test statistic that works in high dimensions, on learned feature representations, and for any data type where a distance can be defined. The authors demonstrate that PTED scales linearly with dimensions and sample size while retaining strong discriminative power, and show it outperforms other multi‑dimensional tests in sensitivity.