We introduce VINE: a non-parametric mutual information (MI) estimator based on simplified D-vine copulas. Approximating the joint density with a simplified D-vine allows the density ratio to simplify...
arXiv:2607. 03487v1 Announce Type: cross Abstract: Mutual information (MI) estimation is a central problem in machine learning and statistics; however, existing benchmarks typically evaluate estimators on simplified, low-dimensional distributions, leaving their performance on complex, realistic data largely unexplored.
ALICE is a foundation model that estimates mutual information (MI) without per‑distribution training. Trained only on synthetic distributions, it acts as an in‑context estimator of rectified‑flow velocity fields, producing MI via a fixed identity that integrates squared differences between joint and conditional fields. The authors validate ALICE on a challenging benchmark and demonstrate its applicability to unseen data in biology, genetics, and neuroscience, achieving performance comparable to neural estimators trained separately for each distribution.
arXiv:2511. 18945v4 Announce Type: replace Abstract: We propose a fully data-driven approach to designing mutual information (MI) estimators.
arXiv:2607. 27710v1 Announce Type: new Abstract: Mutual information is a general measure of statistical dependence that captures both linear and nonlinear relationships between random variables.
arXiv:2606. 16411v1 Announce Type: new Abstract: The Jensen-Shannon divergence is widely reported as a scalar measure of fidelity for synthetic tabular data.
The paper introduces Spread Mutual Information (SMI), a new metric that integrates mutual information over noise levels using a spreading kernel. By applying Gaussian spreading, SMI ensures smooth, strictly positive conditional and marginal densities, enabling gradient estimation even for originally singular distributions. Experiments show that SMI consistently outperforms other MI-based methods for controlling dependence in implicit generative models and competes well with task‑specific approaches.
arXiv:2607. 26955v1 Announce Type: cross Abstract: Minimax-optimal rates for multivariate distribution estimation are known to suffer from the curse of dimensionality.
arXiv:2610.23404v1 Announce Type: new Abstract: Information-theoretic measures, such as Mutual Information (MI), play a crucial role in understanding non-linear relationships between random variables...
arXiv:2606. 00241v1 Announce Type: cross Abstract: Measuring statistical dependency between high-dimensional random variables is a fundamental task in data science and machine learning.
The paper addresses the challenge of estimating joint probability distributions in high-dimensional settings by factorizing them autoregressively and using nonparametric conditional estimators. It establishes theoretical convergence bounds that link the difficulty of joint estimation to the hardest factor in the factorization, especially when conditional estimators adapt to sparsity in an undirected dependence graph. Empirical results show that an autoregressive estimator built with parallel gradient boosting achieves competitive performance on simulated and real datasets, even without optimizing factorization order.
Vine copulas provide a flexible framework for modeling complex multivariate distributions through a hierarchical decomposition into bivariate pair-copulas. Fitting a D-vine requires selecting a copula family and parameter configuration for each pair-copula from a set of candidates encoding different dependence patterns.