The article argues that on AI‑optimised NVIDIA B300 GPUs and newer, the FP8 tensor‑core matrix operation—implemented via the CRT‑based Ozaki Scheme II—can become the primary substrate for matrix‑heavy FP64 kernels while maintaining FP64‑grade accuracy. It introduces the Tensor‑Memory Equilibrium (TME) model, a Roofline extension with four parameters, to show that FP8 can match native FP64 performance under certain intensity thresholds and tile‑fusion conditions. The study identifies two notable exceptions—large dense‑square DGEMM and 3‑D FFT—where additional hardware or software adjustments are required to reach the memory roof. whyItMatters":"The paper demonstrates that FP8, with appropriate reconstruction and deconstruction strategies, can replace native FP64 for high‑performance computing workloads on modern GPUs, potentially reducing hardware complexity and energy consumption while preserving accuracy."
arXiv:2606. 06510v1 Announce Type: cross Abstract: Conventional HPC dogma holds that native hardware FP64 silicon is the irreducible foundation of scientific computing -- the "holy grail" of double-precision simulation.
arXiv:2606. 06510v2 Announce Type: replace-cross Abstract: Conventional HPC holds that native hardware FP64 is the irreducible foundation of scientific computing.
The paper presents a design for executing a full‑FP64 1024³ 3‑D FFT on NVIDIA’s Blackwell Ultra (B300) GPU using FP8 tensor cores. It replaces traditional FP64 arithmetic with a sequence of FP8‑tensor DFT GEMMs, Karatsuba‑based residue combination, and exact CRT reconstruction, leaving only a final conversion for rounding. The main bottleneck identified is a per‑output integer epilogue that limits performance to 63–87 ms, far above the theoretical 12.9 ms roof, and the authors propose modest hardware changes—such as an INT8 tensor core and cross‑column accumulation—to reduce this gap.
The paper identifies a missing cost term in the Tensor‑Memory Equilibrium (TME) model for emulating fp64 on GPUs: the per‑input deconstruction cost required before matrix multiplication can proceed. By adding this term and calibrating it against cuBLAS, the authors derive a closed‑form operational‑intensity threshold (OI*) below which emulation cannot match native fp64 performance. On an NVIDIA B300 GPU, this threshold is about 0.56 FLOP/B, limiting the speed‑ups of memory‑bound kernels such as GEMV, SpMV, and low‑batch GEMV to 0.3–0.9× native, and reducing the 7‑point stencil speed‑up from 3.1× to 1.8×, while dense GEMM remains unaffected.
arXiv:2606. 23698v1 Announce Type: cross Abstract: NVIDIA's Blackwell Ultra (B300) cuts FP64 vector throughput to ~1.
arXiv:2607. 14568v1 Announce Type: cross Abstract: A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.
arXiv:2608. 10103v1 Announce Type: cross Abstract: High-performance Tensor Core kernels rely on a low-level PTX pipeline built from asynchronous data movement with cp.
The paper addresses the problem of non‑deterministic outputs from large language models (LLMs) when run on different GPU architectures, caused by floating‑point non‑associativity and hardware‑dependent kernel choices. It proposes a set of fixed‑configuration fused‑upcast GEMM kernels that load 16‑bit weights, upcast to FP32, and perform IEEE‑754 compliant reductions in a problem‑shape‑dependent order, ensuring identical linear‑layer outputs across NVIDIA Ampere, Ada, and Hopper GPUs. The new approach achieves 1.17–3.1× faster end‑to‑end performance than existing solutions and halves weight‑memory traffic while maintaining cross‑architecture reproducibility.
arXiv:2611.05419v1 Announce Type: cross Abstract: The RTX 2080 Ti (TU102) ships full second-generation Tensor Cores in hardware, yet the vendor throttles FP16 matrix multiplication at the driver leve...
arXiv:2611.07496v1 Announce Type: cross Abstract: Attention is a core building block of frontier models and often dominates compute-bound prefill at long context. FP8 matrix instructions offer twice...
arXiv:2606. 06521v1 Announce Type: cross Abstract: FP8 (E4M3) acceleration for attention computation offers significant throughput gains, but the 3-bit mantissa introduces precision challenges when the softmax probability matrix P is cast to FP8 before the P*V matrix multiplication.
The paper introduces format‑aware fusion, a method that co‑designs quantization producers with their scale domains and consumer layouts to fully exploit four‑bit floating‑point (FP4) Tensor Cores. Using this approach, the authors pretrain the Llama‑3‑family 8B model on 160 billion tokens, achieving up to 37.9 K tokens/s/GPU—significantly higher than standard bfloat16 or Transformer Engine FP4 baselines. The study demonstrates that FP4 performance depends on the interplay of scaling, operand packing, layout, and execution path, with downstream task rankings diverging from training‑loss rankings.