The paper discusses tensorizing neural networks by reshaping dense weight matrices into higher-order tensors and approximating them with low-rank tensor network decompositions. This approach offers promising model compression and introduces bond indices that create new latent spaces, potentially enhancing interpretability. Despite encouraging empirical results, tensorized neural networks remain underused, and the authors call for more research to address practical scaling and adoption challenges.
By Safa Hamreras, Sukhbinder Singh, Rom\'an Or\'us
The article offers an expository overview of why higher‑arity tensor operations matter for deep learning. It presents an empirical study of higher‑arity phenomena in trained neural networks, introduces a hypergraphical generalization of the multilayer perceptron, and examines links to evolutionary algorithms. The paper concludes by outlining promising future research directions.
By Michael L. Roberts, Carlos Zapata Carratal\'a. Nicholas J. Cooper, Lijun Chen, Fran\c{c}ois G. Meyer, Danna Gurari
arXiv:2606. 29043v1 Announce Type: new Abstract: Sharpness and complexity are two central factors in the generalization analysis of deep neural networks.
By Ziyu Cheng, Xitong Zhang, Longxiu Huang, Rongrong Wang
Over the past decade, deep neural networks (DNNs) have achieved remarkable success on complex machine-learning tasks, yet the theoretical foundations of their performance remain incomplete. From a statistical viewpoint, a natural question is: can DNNs attain feature-learning and prediction consistency comparable to that of classical models?
arXiv:2606. 06624v1 Announce Type: new Abstract: In the current era of deep learning and especially generative models, there is significant investment in training very large generative models.
By San Buchanan, Druv Pai, Peng Wang, Yi Ma
arXiv:2607. 05546v1 Announce Type: cross Abstract: We develop a unified function space theory of deep fully connected neural networks.
By Julia Nakhleh, Robert D. Nowak
arXiv:2503. 07325v2 Announce Type: replace Abstract: Understanding and certifying the behavior of modern deep neural networks remains a fundamental challenge in reliable machine learning.
By Khoat Than, Dat Phan
The paper introduces the Universal Weight Encoder (UWE), a scalable representation for weight-space learning that uses a typed graph of interfaces and modules to respect parameter symmetries across diverse neural network architectures. UWE combines local equivariant updates with global attention over invariant interface summaries, enabling a single encoder to process multiple supported architectures efficiently. Experiments show UWE matches or surpasses architecture‑specific state‑of‑the‑art methods on standard benchmarks and excels on a large zoo of 1,200 transformers, ranking unseen architectures accurately while using significantly less memory.
By Micha{\l} Kozyra, Michael Bronstein
The paper introduces a graphical notation, adapted from Penrose tensor notation, to design and represent interpretable AI architectures. This notation provides a global view of an architecture and maps directly onto PyTorch einsum code, enabling clear depiction of tensor manipulations. The authors apply the notation to several interpretable models—concept bottlenecks, sparse probes, prototype networks, neural additive models, and mixtures of linear models—and use it to diagram the key components of the Steerling-8B language model, revealing its residual structure and allowing a concise 33‑line PyTorch implementation.
By Pietro Barbiero
The paper proposes a new strategy for merging layers in deep neural networks, enabling depth compression without requiring an analytical solution for convolutions with padding and without increasing kernel size. This approach addresses limitations of previous methods that struggled with padded convolutions and larger kernels, and it is validated across various architectures and datasets with measured inference speed-ups on embedded platforms.
By Petro Shulzhenko, Gabriele Spadaro, Enzo Tartaglione
arXiv:2606. 21497v2 Announce Type: replace-cross Abstract: Modern deep neural networks are trained using error backpropagation, which requires sequential forward and backward computations across network layers.
By Neeraj Mohan Sushma, Aditya Nagarsekar, Cabrel Teguemne Fokam, Robin Schiewer, Amit Kumar Pal, Anand Subramoney, David Kappel
The paper introduces a pointwise generalization theory for fully connected deep neural networks, using a pointwise Riemannian Dimension derived from eigenvalues of learned feature representations across layers. This framework provides hypothesis-dependent, representation-aware generalization bounds that are significantly tighter than traditional size- or norm-based approaches, both theoretically and experimentally. The authors analytically identify structural properties that explain deep networks’ tractability and empirically show that the pointwise Riemannian Dimension captures feature compression, over‑parameterization effects, and optimizer bias.
By Shaojie Li, Yunbei Xu