arXiv Machine Learning

Testing For Distribution Shifts with Conditional Conformal Test Martingales

arXiv:2602. 13848v2 Announce Type: replace Abstract: We propose a sequential test for detecting arbitrary distribution shifts that allows conformal test martingales (CTMs) to work under a fixed, reference-conditional setting.

arXiv Machine Learning
Jul 13

Global Sequential Testing for Multi-Stream Auditing

arXiv:2602. 21479v3 Announce Type: replace-cross Abstract: Across many risk-sensitive areas, it is critical to continuously audit machine learning systems as we receive more data to quickly determine if they are performing as designed.

By Beepul Bharti, Ambar Pal, Jeremias Sulam
arXiv Machine Learning
Sep 23

Conditional Distributional Treatment Effects: Doubly Robust Estimation and Testing

The paper introduces a new estimand for conditional distributional treatment effects that captures how treatments influence the entire outcome distribution, including variance and tail risks, in a covariate-dependent manner. It presents a doubly robust estimator that is minimax optimal locally and uses it to construct a test for global homogeneity of conditional potential outcome distributions. The test accommodates discrepancies beyond the maximum mean discrepancy, guarantees valid type‑1 error, is consistent against fixed alternatives, and includes a computationally efficient, permutation‑free algorithm with exact closed‑form expressions for two natural discrepancies.

By Saksham Jain, Alex Luedtke
arXiv Statistics ML
Aug 25

Change Detection in Probability Flow ODE: Online Testing in Diffusion Latent Spaces

The paper introduces a sequential change‑point detection method for time‑ordered data where neither the pre‑ nor post‑change distributions have closed forms. It trains a conditional diffusion model on pre‑change data, uses its probability flow ODE to map observations to a Gaussian latent space, and then applies the Maximum Mean Discrepancy as a test statistic. The authors derive closed‑form components under the Gaussian null, establish the statistic’s asymptotic distribution as a degenerate U‑statistic, and implement an online Shiryaev–Roberts procedure with exact threshold calibration to detect arbitrary distributional shifts without parametric assumptions.

By Artem Kraevskiy, Artem Prokhorov