arXiv Statistics ML October 28, 2026 By Johan Mylius-Kroken, Elisabeth Wetzer, Ali Ramezani-Kebrya, Robert Jenssen, Kristoffer Knutsen Wickstr{\o}m

Intrinsic Information-Theoretic Analysis of ReLU Nets

Read the original on arXiv Statistics ML →

arXiv:2610. 36864v1 Announce Type: new Abstract: Mutual information (MI) is fundamental to representation-learning analysis in deep networks, but in high dimensions MI estimators depend on hyperparameters whose bias often dominates the signal.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Statistics ML.

safety

arXiv AI

Oct 5

Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability

arXiv:2610.03000v1 Announce Type: cross Abstract: Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training with...

By Ambarish Moharil

rag

arXiv Machine Learning

Aug 26

Multi-Source Complex Network Reconstruction via Wasserstein Distributionally Robust Optimization and Algorithm Unrolling

The paper introduces MS‑WDRO, a multi‑source Wasserstein distributionally robust optimization framework for reconstructing complex network topologies from scarce target‑domain data and abundant heterogeneous source data. It fuses sources via a weighted Wasserstein barycenter, builds an ambiguity set around it, and solves a regularized Laplacian estimator using a provably convergent ADMM scheme. The authors provide finite‑sample guarantees, demonstrate that naive aggregation is suboptimal, and show through experiments on synthetic data and the ABIDE I neuroimaging dataset that MS‑WDRO outperforms seven baselines in graph recovery, sample efficiency, and diagnostic utility, especially when target samples are limited.

By Chuansen Peng, Yifan Xia, Jinshan Zhong, Xiaojing Shen

efficiency benchmarks safety

arXiv Machine Learning

Oct 21

Polynomial Learning of Positive ReLU Networks under Gaussian Inputs

arXiv:2610. 25116v2 Announce Type: replace Abstract: We study positive, bias-free, one-hidden-layer ReLU networks under standard Gaussian inputs.

By Zhijie Zheng

safety

arXiv Machine Learning

Jul 7

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

arXiv:2607. 03613v1 Announce Type: new Abstract: We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression.

By Shuang Liang, Tom Jacobs, Guido Mont\'ufar

safety

arXiv Machine Learning

Jun 16

The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints

arXiv:2606. 16028v1 Announce Type: new Abstract: Modern deep learning architectures are increasingly multi-task and multi-modal, using a pretrained foundation model combined with task-specific, fine-tuned models.

By Thomas Dittrich, Oliver Potocki, Philipp Grohs

fine-tuning multimodal

Hugging Face Trending Papers

Oct 26

What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control

The information bottleneck (IB) trades compression of an input $X$ against prediction of a target $Y$ through a multiplier $β=1/λ$, yet each operating point is usually served by training, or fully con...

safety

One email a morning, machine-written

Related stories

Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability

Multi-Source Complex Network Reconstruction via Wasserstein Distributionally Robust Optimization and Algorithm Unrolling

Polynomial Learning of Positive ReLU Networks under Gaussian Inputs

Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints

What Should an Information Bottleneck Encoder Learn? Separating Representation Learning from Rate Control