arXiv AI

CircuitTree: Composing Circuits into A Syllable Transcriber

arXiv:2610. 12520v1 Announce Type: cross Abstract: The goal of circuit discovery is to decompose a monolithic neural network into tiny modular subnetworks which are interpretable in the sense that each does an identifiable subtask.

arXiv AI
4d ago

Certified Circuits: Stability Guarantees for Mechanistic Circuits

Certified Circuits introduces a framework that wraps any black‑box circuit discovery algorithm with randomized data subsampling to provide provable stability guarantees. By certifying that inclusion decisions over circuit components are invariant to bounded edit‑distance perturbations of the concept dataset, unstable components are abstained from, producing more compact and accurate mechanistic explanations. Across ResNet, ViT, and GPT‑2 on vision and language tasks, Certified Circuits achieves up to 56% higher accuracy and up to 80% fewer components, maintaining reliability where baseline methods degrade.

By Alaa Anani, Tobias Lorenz, Bernt Schiele, Mario Fritz, Jonas Fischer
arXiv AI
Sep 2

Building Expressive and Tractable Probabilistic Generative Models: A Review

The article surveys recent progress in tractable probabilistic generative modeling, with a focus on Probabilistic Circuits (PCs). It offers a unified view of the trade‑offs between expressivity and tractability, outlining design principles, algorithmic extensions, and a taxonomy of the field. The review also covers deep and hybrid PCs that integrate ideas from deep neural models, and highlights challenges and open questions for future research.

By Sahil Sidheekh, Sriraam Natarajan
arXiv Machine Learning
Jun 3

Fast and Expressive Multi-Byte Prediction with Probabilistic Circuits

arXiv:2511. 11346v2 Announce Type: replace Abstract: Multi-token prediction (MTP) is a prominent strategy to significantly speed up generation in large language models (LLMs), especially in byte-level LLMs, which are tokeniser-free but prohibitively slow.

By Andreas Grivas, Lorenzo Loconte, Emile van Krieken, Piotr Nawrot, Yu Zhao, Euan Wielewski, Pasquale Minervini, Edoardo Ponti, Antonio Vergari