arXiv:2611.07693v1 Announce Type: new Abstract: We introduce VINE: a non-parametric mutual information (MI) estimator based on simplified D-vine copulas. Approximating the joint density with a simpli...
arXiv:2607. 03487v1 Announce Type: cross Abstract: Mutual information (MI) estimation is a central problem in machine learning and statistics; however, existing benchmarks typically evaluate estimators on simplified, low-dimensional distributions, leaving their performance on complex, realistic data largely unexplored.
ALICE is a foundation model that estimates mutual information (MI) without per‑distribution training. Trained only on synthetic distributions, it acts as an in‑context estimator of rectified‑flow velocity fields, producing MI via a fixed identity that integrates squared differences between joint and conditional fields. The authors validate ALICE on a challenging benchmark and demonstrate its applicability to unseen data in biology, genetics, and neuroscience, achieving performance comparable to neural estimators trained separately for each distribution.
Vine copulas provide a flexible framework for modeling complex multivariate distributions through a hierarchical decomposition into bivariate pair-copulas. Fitting a D-vine requires selecting a copula family and parameter configuration for each pair-copula from a set of candidates encoding different dependence patterns.
arXiv:2607. 26955v1 Announce Type: cross Abstract: Minimax-optimal rates for multivariate distribution estimation are known to suffer from the curse of dimensionality.
arXiv:2607. 25020v1 Announce Type: new Abstract: Vine copulas provide a flexible framework for modeling complex multivariate distributions through a hierarchical decomposition into bivariate pair-copulas.
The paper introduces Spread Mutual Information (SMI), a new metric that integrates mutual information over noise levels using a spreading kernel. By applying Gaussian spreading, SMI ensures smooth, strictly positive conditional and marginal densities, enabling gradient estimation even for originally singular distributions. Experiments show that SMI consistently outperforms other MI-based methods for controlling dependence in implicit generative models and competes well with task‑specific approaches.
arXiv:2511. 18945v4 Announce Type: replace Abstract: We propose a fully data-driven approach to designing mutual information (MI) estimators.
arXiv:2610.23404v1 Announce Type: new Abstract: Information-theoretic measures, such as Mutual Information (MI), play a crucial role in understanding non-linear relationships between random variables...
The paper introduces Dirichlet Process Mixtures (DPMs) as a new non‑parametric conditional probability distribution (CPD) model for Semiparametric Bayesian Networks (SPBNs). By replacing kernel density estimators with DPMs, the authors achieve a more compact representation that reduces inference costs and improves density estimation accuracy. Experiments on synthetic and real‑world data show that DPM‑based SPBNs outperform existing SPBNs and other continuous‑variable Bayesian network models in both density estimation and structure learning tasks.
arXiv:2607. 27710v1 Announce Type: new Abstract: Mutual information is a general measure of statistical dependence that captures both linear and nonlinear relationships between random variables.
The paper presents Copula Adapted Directed Acyclic Graph (CopDAG), a framework that combines copula models with an ensemble of causal structure discovery methods based on Directed Acyclic Graphs to represent biomedical data. By capturing non‑Gaussian, non‑linear dependencies and stable causal relationships, CopDAG enables clustering of unlabeled biomedical data using K‑means. Across 16 biomedical datasets, CopDAG achieves the highest normalized clustering accuracy and adjusted Rand index among 12 evaluated methods, and it can predict class labels and provide explainable causal visualizations without relying on data annotations.