arXiv:2610. 36864v1 Announce Type: new Abstract: Mutual information (MI) is fundamental to representation-learning analysis in deep networks, but in high dimensions MI estimators depend on hyperparameters whose bias often dominates the signal.
arXiv:2610.03000v1 Announce Type: cross Abstract: Intrinsic explainability remains a challenging problem, particularly in contexts where multilayer perceptrons (MLPs) require dynamic re-training with...
The paper introduces MS‑WDRO, a multi‑source Wasserstein distributionally robust optimization framework for reconstructing complex network topologies from scarce target‑domain data and abundant heterogeneous source data. It fuses sources via a weighted Wasserstein barycenter, builds an ambiguity set around it, and solves a regularized Laplacian estimator using a provably convergent ADMM scheme. The authors provide finite‑sample guarantees, demonstrate that naive aggregation is suboptimal, and show through experiments on synthetic data and the ABIDE I neuroimaging dataset that MS‑WDRO outperforms seven baselines in graph recovery, sample efficiency, and diagnostic utility, especially when target samples are limited.
arXiv:2610. 25116v2 Announce Type: replace Abstract: We study positive, bias-free, one-hidden-layer ReLU networks under standard Gaussian inputs.
arXiv:2607. 03613v1 Announce Type: new Abstract: We study the implicit bias of noisy stochastic gradient descent in training wide two-layer ReLU networks for multivariate regression.
arXiv:2606. 16028v1 Announce Type: new Abstract: Modern deep learning architectures are increasingly multi-task and multi-modal, using a pretrained foundation model combined with task-specific, fine-tuned models.
The information bottleneck (IB) trades compression of an input $X$ against prediction of a target $Y$ through a multiplier $β=1/λ$, yet each operating point is usually served by training, or fully con...
arXiv:2610. 23007v1 Announce Type: new Abstract: Gradient descent (GD) with large learning rates can successfully optimize neural networks even though classical smoothness-based learning rate conditions are violated.
arXiv:2608. 09707v1 Announce Type: cross Abstract: Embedding trained neural networks as surrogates within optimisation problems is an established practice in operations research.
The paper investigates the geometry and closure properties of function spaces realized by ReLU neural networks with fixed architecture. It proves that the minimal number of convex pieces in continuous piecewise-linear functions is lower semicontinuous under pointwise convergence, and uses this to show that shallow scalar ReLU realization spaces are closed in the weighted supremum norm, while providing a counterexample that demonstrates non-closedness for a specific architecture. The work also characterizes the role of a blow‑up residue in non‑closedness and offers new geometric and functional‑analytic tools for analyzing ReLU realization spaces.
The paper investigates minimal‑norm interpolation and λ2‑regularized logistic‑loss minimization for binary classification using univariate two‑layer ReLU networks. It provides exact geometric characterizations of optimal classifiers, showing that unpenalized hidden‑layer biases yield continuous piecewise‑affine functions that tightly follow label switches, while penalized biases produce a unique, sparsest classifier with a single kink per same‑label segment. Adding a free affine skip connection does not change these function‑space solutions but guarantees that every KKT point becomes globally optimal, eliminating suboptimal KKT points that can arise without the skip connection.
arXiv:2608. 15472v1 Announce Type: cross Abstract: The problem of networked information aggregation, studied in Kearns et al.
arXiv:2607. 08843v1 Announce Type: new Abstract: In artificial and biological neural networks, concepts are often encoded as consistent linear directions in representation space.