arXiv:2502.04312v3 Announce Type: replace Abstract: Contrastive learning leverages data augmentation to develop feature representation without relying on large labeled datasets. However, despite its...
The paper introduces Neural Low-Degree Filtering (Neural LoFi), a stylized limit of gradient-based training that turns hierarchical feature learning into an explicit iterative spectral procedure. In this framework, each layer independently selects directions with maximal low-degree correlation to the label, providing a tractable surrogate for deep learning and a kernel-space interpretation. Experiments on fully connected and convolutional networks show that Neural LoFi outperforms lazy random-feature baselines, recovers meaningful structured filters, and aligns with early gradient-descent feature discovery on real datasets.
arXiv:2604. 20308v2 Announce Type: replace Abstract: Graph neural networks face two fundamental challenges rooted in the linear structure of Euclidean vector spaces: (1) Current architectures represent geometry through vectors (directions, gradients), yet many tasks require matrix-valued representations that capture relationships between directions-such as how atomic orientations covary in a molecule.
arXiv:2606. 14334v1 Announce Type: new Abstract: High-dimensional datasets often concentrate near low-dimensional structures, but estimating their geometry from samples typically relies on graphs and kernels that scale poorly with dataset size and dimension.
arXiv:2608. 01318v1 Announce Type: cross Abstract: Modern sensing, communication, and learning systems generate heterogeneous network signals, with local data differing in dimension, modality, and geometric structure.
LocaLeJEPA extends Joint-Embedding Predictive Architectures (JEPA) by introducing a manifold-theoretic reformulation that aligns representation learning with non‑Euclidean geometric priors. It incorporates a locally Euclidean distribution‑matching regularizer, LocaLeReg, to stabilize training and scale to large models. Experiments on ImageNet‑1K show that LocaLeJEPA outperforms the original LeJEPA across both hyperbolic and Euclidean backbones, improving linear and k‑NN classification as well as downstream tasks.
Modern sensing, communication, and learning systems generate heterogeneous network signals, with local data differing in dimension, modality, and geometric structure. Processing such data requires a mathematical framework capable of simultaneously modeling heterogeneous local signal spaces and the transformations relating them.
arXiv:2505. 19809v3 Announce Type: replace-cross Abstract: In many real-world applications of regression, conditional probability estimation, and uncertainty quantification, exploiting symmetries rooted in physics or geometry can dramatically improve generalization and sample efficiency.
ChronoSheaf introduces a joint spatio-temporal spectral filtering framework for cellular sheaves, extending sheaf neural networks beyond instantaneous spatial operators. It defines a spectral sheaf Laplacian that simultaneously governs signal propagation across both graph structure and time, and employs a Dual Spectral Filter to broaden receptive fields without enlarging the sheaf. The model incorporates Transitional Restriction Maps to allow asymmetric temporal influence and demonstrates superior performance on various spatio-temporal forecasting benchmarks.
arXiv:2606. 03270v1 Announce Type: cross Abstract: Foundation models have sparked a revolution via a pretraining-adaptation paradigm, with recent efforts extending this success to graphs.
arXiv:2608. 10016v1 Announce Type: cross Abstract: Heterogeneous federated systems require agents to learn and exchange informative representations despite differences in data distributions, sensing modalities, model architectures, latent dimensionalities, and local learning objectives.
Sheaf neural networks generalize graph neural networks by assigning each edge a learned linear restriction map that specifies how the local representations of adjacent nodes relate to one another, the...