arXiv Machine Learning

Numbat: Building and Verifying a Self-Contained Machine-Learning Stack

The paper introduces Numbat, a self‑contained machine‑learning stack implemented entirely in Zig with no external runtime dependencies. It covers tensor computation, automatic differentiation, neural‑network modules, mixed precision, multi‑GPU training, data loading, and monitoring, and exposes a stable C ABI with over 1,400 entry points and bindings for six languages. The authors verify the stack against a reference implementation at multiple levels, uncovering ten silent recipe divergences, and demonstrate its practical capability by training a 25.9M‑parameter YOLOv8m detector on COCO 2017, achieving a competitive mAP score and matching single‑GPU performance.

arXiv Computation and Language
Oct 19

Automatic Tensor-level Execution of Abstract Interpreters for Neural Networks

Neuton is a framework that automatically converts neuron‑level abstract interpreter specifications into tensor‑level executables, enabling the creation of new DNN certifiers without manual tensor programming. It introduces a stack‑based intermediate representation with shape analysis, domain‑specific rewrite rules for efficient tensor operations, and a novel sparse tensor format (gBCSR) tailored to certifier sparsity. The authors demonstrate that Neuton preserves neuron‑level semantics and evaluate it on eight certifiers across 32 DNN benchmarks, compiling four existing certifiers and producing executable implementations for four others, revealing new precision‑runtime trade‑offs such as SkipPoly matching DeepPoly’s precision while often running faster.

By Avaljot Singh, Yasmin Chandini Sarita, Aditya Mishra, Gagandeep Singh, Charith Mendis
arXiv Machine Learning
Sep 16

OPEN-1B: A Fully Auditable Training Run

The paper introduces Open-1B, a language model trained under a new fully auditable regime that ensures every training operation is reproducible on heterogeneous commodity hardware with bitwise certainty. By enforcing a fixed order on sources of nondeterminism—GPU reductions, data batch ordering, and inter/intra-node communication—the authors enable auditors to replay and verify individual training steps on a single machine. The release includes the full pretraining dataset, all intermediate checkpoints, the training codebase, and an audit harness for step-by-step verification.

By John Donaghy, Brian Wilcox, O\u{g}uzhan Ersoy, Shikhar Rastogi, Adam St Arnaud, Alexey Titov, Jordan Greenberg, Ben Fielding, Harry Grieve
arXiv Computation and Language
Oct 23

SCOPE: Compiling Natural-Language Policies into Neurosymbolic Predicate Programs

SCOPE compiles natural‑language policies into neurosymbolic predicate programs that combine small neural models, deterministic extractors, and logical composition. The system is jointly optimized with adversarially generated counterexamples, achieving higher accuracy than prompt‑optimization, task‑decomposition, and workflow‑generation baselines on real‑world policy benchmarks. Programs trained with the weakest executor transfer to other models, enabling deployment on inexpensive models with stable, interpretable predictions.

By Arnon Mazza, Elad Levi
arXiv Machine Learning
Oct 16

Stateful Testing for Deep Learning Compilers

The paper introduces Rondo, a stateful testing framework for deep‑learning compilers that models each test as a model plus an invocation sequence. Rondo mutates sequences using a Markov chain Monte Carlo sampler guided by compilation behavior, and employs a three‑way differential oracle to detect bugs that arise only when compiler state is retained. Evaluated on four major frameworks and 24 model architectures, Rondo uncovered 117 bugs—35 of which were stateful—leading to 39 bug reports, 35 confirmed by developers, and 18 fixes, while achieving high coverage and precision.

By Jiatao Ge, Ruixiang Qian, Yibo Hu, Yanzhou Mu, Yinglong Zou, Jiawei Liu, Zhenyu Chen, Chunrong Fang
arXiv Machine Learning
Aug 4

Nova: An End-to-End MLIR Compiler for Deep Learning

arXiv:2608. 00029v1 Announce Type: cross Abstract: The performance of deep learning models at scale relies heavily on how effectively high-level mathematical operations are mapped to underlying physical hardware.

By Adwaid Suresh, Aparna A, Harshini V M, Jona Delcy C A, Killi Uma Maheswara Rao, Ram Charan Golla, Surendra Vendra
arXiv AI
Oct 23

Proposals and challenges for zero-knowledge verification of frontier AI training

The paper proposes a zero‑knowledge verification framework for frontier AI training, combining pre‑committed specifications, network observations, and live Merkle commitments to create a succinct proof layer via a zero‑knowledge virtual machine (zkVM). It outlines how proofs can verify BF16/FP32 semantics step‑by‑step, enforce policy‑relevant claims, and provide a governance‑auditable training record, while also discussing sampling guarantees, weight smuggling risks, and the relative robustness of global training budgets versus per‑run FLOP caps. The authors compare zkVMs, custom circuits, and interactive audits, and identify remaining research and engineering challenges for external contributors.

By Pierre Peign\'e, Ky Nguyen, Paul Wang
arXiv AI
5d ago

ProCerDCL: Query-Efficient Black-Box Probabilistic Verification of Deep Learning Compiler Optimization

ProCerDCL is a query‑efficient framework for black‑box probabilistic verification of deep learning compiler optimizations. It uses a single sequential test to decide whether the violation probability of a binary property oracle is below a threshold, above a higher threshold plus a margin, or inconclusive, with bounded error probabilities. Applied to TorchInductor and TensorRT, it achieves higher sampling efficiency and provides diagnostic verdicts on model semantic consistency across 24 tasks.

By Zexin Li, Weibo Zhang, Linyi Li, Arvind Easwaran, Simin Chen