The Noise Floor of a Quantization Comparison: Engine, restart, batch and kernel drift, measured against 8-bit and 4-bit effects in two decision models
Read the original on Hugging Face Trending Papers →The paper investigates how various non‑quantization factors—such as engine changes, server restarts, batching differences, and kernel selection—affect the performance of quantized language models compared to their BF16 baselines. Using 2,400 public decisions from Cloudflare’s Clef‑flash (9B) and Clef (27B) models on an NVIDIA GB10, the authors quantify the noise introduced by each factor and isolate the true impact of different quantization schemes (8‑bit, FP8, Q4_K_M, NVFP4). They find that non‑quantization sources contribute up to 0.92% of decision changes, while 4‑bit quantization can alter accuracy by up to 0.95 points, and they provide code, recipes, and detailed per‑record outputs for reproducibility.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.