Graphical Design of Interpretable Architectures

The paper introduces a graphical notation, adapted from Penrose tensor notation, to design and represent interpretable AI architectures. This notation provides a global view of an architecture and maps directly onto PyTorch einsum code, enabling clear depiction of tensor manipulations. The authors apply the notation to several interpretable models—concept bottlenecks, sparse probes, prototype networks, neural additive models, and mixtures of linear models—and use it to diagram the key components of the Steerling-8B language model, revealing its residual structure and allowing a concise 33‑line PyTorch implementation.