Introducing Training Cluster as a Service - a new collaboration with NVIDIA

From Detection to Recovery: Operational Analysis on LLM Pre-training with 504 GPUs

arXiv:2605. 09370v3 Announce Type: replace-cross Abstract: Large-scale AI training is now fundamentally a distributed systems problem, and hardware failures have become routine operating conditions rather than rare exceptions.

OpenAI and NVIDIA announce strategic partnership to deploy 10 gigawatts of NVIDIA systems

OpenAI and NVIDIA announce a strategic partnership to deploy 10 gigawatts of AI datacenters powered by NVIDIA systems, with the first phase launching in 2026.

Easily Train Models with H100 GPUs on NVIDIA DGX Cloud

Mistral AI partners with NVIDIA to accelerate open frontier models

Faster Training and Inference: Habana Gaudi®2 vs Nvidia A100 80GB

Leto: Fast In-Place Recovery for LLM Training on Surviving Hardware

Leto is a fault‑tolerant training system for large language models that enables fast in‑place recovery on surviving hardware after hardware‑operable failures. It retains the working model state and reusable process state, while pre‑initializing remaining state in a shadow trainer, using two‑tier erasure protection and chunk‑level transactional updates to maintain consistency. Experiments on NVIDIA A100 clusters show Leto recovers 3.6–6.5× faster than checkpointing baselines and boosts productive training time by up to 13.7 percentage points, with simulations indicating over 95% productivity on a 131,072‑GPU cluster.