arXiv Machine Learning
Jun 8
MIST: Mutual Information Estimation Via Supervised Training
arXiv:2511. 18945v4 Announce Type: replace Abstract: We propose a fully data-driven approach to designing mutual information (MI) estimators.
NMINE: Normalized Mutual Information Neural Estimation
arXiv:2607. 27710v1 Announce Type: new Abstract: Mutual information is a general measure of statistical dependence that captures both linear and nonlinear relationships between random variables.
Related stories
arXiv Machine Learning
Jun 18
DIPHINE: Diffusion-based $\Phi$-ID Neural Estimator
arXiv:2606. 18997v1 Announce Type: new Abstract: Uncovering the true informational architecture of real-world complex systems requires disentangling how their components uniquely store, redundantly share, and synergistically integrate information over time.
arXiv AI
Jul 21
Information-Theoretic Measures in AI: A Practical Decision Framework
arXiv:2604. 23716v3 Announce Type: replace Abstract: Information-theoretic (IT) measures are ubiquitous in artificial intelligence: entropy drives decision-tree splits and uncertainty quantification, cross-entropy is the default classification loss, mutual information underpins representation learning and feature selection, and transfer entropy reveals directed influence in dynamical systems.
arXiv AI
Jun 2
Estimating Mutual Information between Time Series and Temporal Event Sequences Across Diverse Analysis Tasks
arXiv:2606. 01602v1 Announce Type: cross Abstract: Pairwise dependence measures such as correlation and causality are fundamental to temporal data mining, yet there is still no principled and robust way to quantify dependence between heterogeneous data types, especially between continuous time series and discrete temporal event sequences.
arXiv Machine Learning
Jun 9
Dendrograms of Mixing Measures for Softmax-Gated Gaussian Mixture of Experts: Consistency Without Model Sweeps
arXiv:2510. 12744v2 Announce Type: replace-cross Abstract: We develop a unified statistical framework for softmax-gated Gaussian mixture of experts (SGMoE) that addresses three long-standing obstacles in parameter estimation and model selection: (i) non-identifiability of gating parameters up to common translations, (ii) intrinsic gate-expert interactions that induce coupled differential relations in the likelihood, and (iii) the tight numerator-denominator coupling in the softmax-induced conditional density.
arXiv Machine Learning
Jul 31
Understanding Submodular Information Measure Based Objectives for Representation Learning: A Variance and Separation Perspective
arXiv:2607. 27660v1 Announce Type: new Abstract: Submodular Information Measures (SIMs) have recently emerged as a powerful framework for representation learning and multimodal learning.
arXiv AI
Jun 2
InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimate
arXiv:2606. 00241v1 Announce Type: cross Abstract: Measuring statistical dependency between high-dimensional random variables is a fundamental task in data science and machine learning.
arXiv Machine Learning
Jun 16
Not all Jensen-Shannon Divergence Estimators are Equal
arXiv:2606. 16411v1 Announce Type: new Abstract: The Jensen-Shannon divergence is widely reported as a scalar measure of fidelity for synthetic tabular data.
Hugging Face Trending Papers
Jul 30
Understanding Submodular Information Measure Based Objectives for Representation Learning: A Variance and Separation Perspective
Submodular Information Measures (SIMs) have recently emerged as a powerful framework for representation learning and multimodal learning. In particular, the SCORE framework~\cite{majee2024score} demonstrated that SIMs can serve as effective objectives for supervised contrastive learning.
arXiv Machine Learning
Jun 16
The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints
arXiv:2606. 16028v1 Announce Type: new Abstract: Modern deep learning architectures are increasingly multi-task and multi-modal, using a pretrained foundation model combined with task-specific, fine-tuned models.
arXiv Machine Learning
Aug 12
FiGuRO: Intrinsic Dimension Estimation for Multi-Modal Data
arXiv:2608. 10857v1 Announce Type: new Abstract: Determining the complexity, or Intrinsic Dimension (ID), of data is fundamental to efficient and interpretable representation learning.
arXiv Machine Learning
Jun 8
Deep Single-Index Fr\'echet Regression
arXiv:2606. 06957v1 Announce Type: cross Abstract: Predicting outputs that are located in non-Euclidean spaces, such as probability distributions, networks, and symmetric positive-definite matrices, is becoming increasingly important in modern data analysis, particularly when inputs are high-dimensional.