The paper introduces a graphical notation, adapted from Penrose tensor notation, to design and represent interpretable AI architectures. This notation provides a global view of an architecture and maps directly onto PyTorch einsum code, enabling clear depiction of tensor manipulations. The authors apply the notation to several interpretable models—concept bottlenecks, sparse probes, prototype networks, neural additive models, and mixtures of linear models—and use it to diagram the key components of the Steerling-8B language model, revealing its residual structure and allowing a concise 33‑line PyTorch implementation.
By Pietro Barbiero
The paper introduces a new graphical notation, adapted from Penrose tensor notation, to design and represent interpretable AI architectures. Unlike symbolic equations or probabilistic models, this notation provides a global view of an architecture while directly mapping to PyTorch einsum code. The authors demonstrate its use on several interpretable models and on the Steerling-8B language model, revealing structural insights and enabling concise code generation.
arXiv:2607. 28755v1 Announce Type: new Abstract: Over the last decade, neural networks have been applied to an increasingly diverse range of applications, including data with rich geometric, topological, or symmetry-related structure.
By Brendan Kennedy, Tegan Emerson, Gregory Roek, Emilie Purvine, Henry Kvinge
arXiv:2608. 13572v1 Announce Type: cross Abstract: We present The Architect, a system that turns Microsoft Excel into an interactive view of deep learning mathematics.
By Mohammad Imrul Jubair, Tom Yeh
arXiv:2608. 11136v1 Announce Type: new Abstract: Logic Tensor Networks (LTN) provide a neurosymbolic framework in which first-order logic is interpreted through tensor operations, enabling logical constraints to be integrated with differentiable learning.
By Davide Rinaldi, Luciano Serafini
The paper discusses tensorizing neural networks by reshaping dense weight matrices into higher-order tensors and approximating them with low-rank tensor network decompositions. This approach offers promising model compression and introduces bond indices that create new latent spaces, potentially enhancing interpretability. Despite encouraging empirical results, tensorized neural networks remain underused, and the authors call for more research to address practical scaling and adoption challenges.
By Safa Hamreras, Sukhbinder Singh, Rom\'an Or\'us