← Back to all news
arXiv Machine Learning By John Donaghy, Brian Wilcox, O\u{g}uzhan Ersoy, Shikhar Rastogi, Adam St Arnaud, Alexey Titov, Jordan Greenberg, Ben Fielding, Harry Grieve

OPEN-1B: A Fully Auditable Training Run

Read the original on arXiv Machine Learning →

The paper introduces Open-1B, a language model trained under a new fully auditable regime that ensures every training operation is reproducible on heterogeneous commodity hardware with bitwise certainty. By enforcing a fixed order on sources of nondeterminism—GPU reductions, data batch ordering, and inter/intra-node communication—the authors enable auditors to replay and verify individual training steps on a single machine. The release includes the full pretraining dataset, all intermediate checkpoints, the training codebase, and an audit harness for step-by-step verification.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv Machine Learning
Sep 30

OVIG: Optimistic Verification of AI Training Integrity via Gradient Signals

OVIG is an optimistic verification framework that audits AI training by replaying the process and comparing gradient differences against an empirically calibrated boundary. It treats any gradient difference exceeding this boundary as a malicious deviation. By partitioning training into stride‑s intervals and storing evidence only at interval endpoints, OVIG dramatically reduces off‑chain storage and transmission costs while maintaining zero attack success rate across language, vision, and diffusion workloads.

By Hongxu Su, Jianzhu Yao, Huan Zhang, Xuechao Wang, Pramod Viswanath
diffusionroboticsmultimodal
More like this →
arXiv AI
Oct 23

Proposals and challenges for zero-knowledge verification of frontier AI training

The paper proposes a zero‑knowledge verification framework for frontier AI training, combining pre‑committed specifications, network observations, and live Merkle commitments to create a succinct proof layer via a zero‑knowledge virtual machine (zkVM). It outlines how proofs can verify BF16/FP32 semantics step‑by‑step, enforce policy‑relevant claims, and provide a governance‑auditable training record, while also discussing sampling guarantees, weight smuggling risks, and the relative robustness of global training budgets versus per‑run FLOP caps. The authors compare zkVMs, custom circuits, and interactive audits, and identify remaining research and engineering challenges for external contributors.

By Pierre Peign\'e, Ky Nguyen, Paul Wang
More like this →
arXiv Machine Learning
Sep 11

Numbat: Building and Verifying a Self-Contained Machine-Learning Stack

The paper introduces Numbat, a self‑contained machine‑learning stack implemented entirely in Zig with no external runtime dependencies. It covers tensor computation, automatic differentiation, neural‑network modules, mixed precision, multi‑GPU training, data loading, and monitoring, and exposes a stable C ABI with over 1,400 entry points and bindings for six languages. The authors verify the stack against a reference implementation at multiple levels, uncovering ten silent recipe divergences, and demonstrate its practical capability by training a 25.9M‑parameter YOLOv8m detector on COCO 2017, achieving a competitive mAP score and matching single‑GPU performance.

By Thang Tran (CloudKites AI Lab, New South Wales, Australia), Lan Dang (Monash Business School, Monash University, Victoria, Australia)
More like this →