← Back to all news
arXiv Statistics ML By Artemy Kolchinsky

Redundancy and synergy in multivariate Gaussians via the Blackwell order

Read the original on arXiv Statistics ML →

The article introduces a partial information decomposition (PID) for multivariate Gaussian systems using the Blackwell order, which determines when one channel is more informative than another. It shows that Gaussian channels optimally extract redundant and union information, providing a geometric interpretation and an efficient numerical algorithm. The authors derive closed‑form expressions for two‑source cases, compare their Blackwell redundancy to the BROJA measures, and demonstrate scalability to systems with up to a thousand dimensions or sources, including an application to an optimal control problem.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Statistics ML.

Related stories

arXiv Statistics ML
Oct 12

Gaussian BROJA Partial Information Decomposition in Closed Form

The paper presents a closed‑form solution for the BROJA partial information decomposition (PID) when all variables are jointly Gaussian. This analytic expression eliminates the need for numerical optimization, allowing direct computation of unique, redundant, and synergistic information components in multivariate Gaussian settings. The resulting formula yields speedups of more than two orders of magnitude for high‑dimensional Gaussian PID calculations.

By Ohad Sheelo, Tal Golan, Ari Pakman
More like this →
arXiv Machine Learning
Oct 22

From Common Information to Canonical Correlation Analysis and Beyond: Bridging Information Theory, Signal Processing, and Machine Learning

The article surveys how the problem of extracting a common component from multiple jointly observed signals has been approached independently by statisticians, information theorists, and engineers. It reviews canonical correlation analysis, common information concepts, and subspace methods, then unifies these perspectives, extending to nonlinear CCA and discussing identifiability and links to self‑supervised learning. The tutorial aims to provide a self‑contained exposition that bridges signal processing, machine learning, and information theory.

By Paris A. Karakasis, Walid Saad
More like this →
arXiv Machine Learning
Jun 18

DIPHINE: Diffusion-based $\Phi$-ID Neural Estimator

arXiv:2606. 18997v1 Announce Type: new Abstract: Uncovering the true informational architecture of real-world complex systems requires disentangling how their components uniquely store, redundantly share, and synergistically integrate information over time.

By Simon Pedro Galeano Munoz, Mustapha Bounoua, Giulio Franzese, Pietro Michiardi, Maurizio Filippone
diffusionbenchmarks
More like this →
arXiv Machine Learning
Jun 5

Inverse Entropic Optimal Transport Solves Semi-supervised Learning via Data Likelihood Maximization

arXiv:2410. 02628v5 Announce Type: replace Abstract: Learning conditional distributions $\pi^*(\cdot|x)$ is a central problem in machine learning, which is typically approached via supervised methods with paired data $(x,y) \sim \pi^*$.

By Mikhail Persiianov, Arip Asadulaev, Nikita Andreev, Nikita Starodubcev, Dmitry Baranchuk, Anastasis Kratsios, Evgeny Burnaev, Alexander Korotin
More like this →