arXiv Machine Learning

Entropy Equivalence Testing

arXiv:2605. 23225v2 Announce Type: replace-cross Abstract: We introduce the problem of \emph{entropy equivalence testing} for probability distributions, a relaxation of the well-studied closeness testing problem, where the distribution testing algorithm is now only required to distinguish, given samples from two unknown distributions $p,q$ and a parameter $\varepsilon \in(0,1/2]$, between $p=q$ and $|H(p)-H(q)| \geq \varepsilon$ (where $H$ denotes the Shannon entropy).

arXiv Statistics ML
Oct 9

Tight Bounds for Equivalence Testing with Non-Adaptive Conditional Samples

The paper presents tight bounds for equivalence testing using non‑adaptive conditional samples. It shows that determining whether two unknown distributions are identical or ε‑far in total variation distance requires ∫Θ((log n)/ε²) queries. Consequently, the query complexity for uniformity, identity, and equivalence testing with non‑adaptive conditional samples is ∫Θ(log n).

By Gautam Kamath
arXiv Statistics ML
Sep 25

All you need is log

arXiv:2606. 27349v3 Announce Type: replace-cross Abstract: How different are several probability distributions from one another?

By Akshay Balsubramani
arXiv Machine Learning
Jul 20

Testing Distributions Against Bounded Distinguishers

arXiv:2607. 15645v1 Announce Type: cross Abstract: Motivated by the challenge of testing distributions over high-dimensional or continuous domains, we study distribution testing with respect to bounded classes of distinguishers.

By Mark Bun, Rathin Desai, Renato Ferreira Pinto Jr
arXiv Machine Learning
Oct 12

Universal Bounds for Self-Normalized Kullback-Leibler Estimation

The paper introduces an entropy-based estimator for Kullback-Leibler divergence that uses normalized importance weights. It establishes a universal finite-sample bound, proving that the estimator’s expectation can exceed the true target‑proposal KL divergence by no more than λ2, regardless of sample size. Additionally, it shows that the KL divergence between the distribution produced by importance‑weighted resampling and the proposal is bounded by both the estimator’s expectation and the target‑proposal KL divergence, without requiring bounded weights or finite higher moments. The authors apply these theoretical results to test‑time alignment of generative models via Soft Best‑of‑N sampling and to relative entropy coding, yielding finite‑sample policy‑divergence guarantees and moment‑free bounds on coding cost.

By Buu Phan, Lester Mackey, David Alvarez-Melis
arXiv Machine Learning
Jul 28

Learning Distributions from Multiple Data Providers

arXiv:2607. 24732v1 Announce Type: cross Abstract: Motivated by learning from heterogeneous and overlapping data providers, we study a stylized model of distribution learning from restricted conditional samples.

By Jon Kleinberg, Amin Saberi, Xizhi Tan, Grigoris Velegkas