Hugging Face Trending Papers

The Noise Floor of a Quantization Comparison: Engine, restart, batch and kernel drift, measured against 8-bit and 4-bit effects in two decision models

Read the original on Hugging Face Trending Papers →

The paper investigates how various non‑quantization factors—such as engine changes, server restarts, batching differences, and kernel selection—affect the performance of quantized language models compared to their BF16 baselines. Using 2,400 public decisions from Cloudflare’s Clef‑flash (9B) and Clef (27B) models on an NVIDIA GB10, the authors quantify the noise introduced by each factor and isolate the true impact of different quantization schemes (8‑bit, FP8, Q4_K_M, NVFP4). They find that non‑quantization sources contribute up to 0.92% of decision changes, while 4‑bit quantization can alter accuracy by up to 0.95 points, and they provide code, recipes, and detailed per‑record outputs for reproducibility.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv Machine Learning
Sep 16

Is INT8 Portable? A Cross-Platform Measurement Study of Quantized Inference on Embedded and Automotive Accelerators

The study evaluates the portability of INT8 post‑training quantization across seven hardware platforms, including CPUs, GPUs, and vendor NPUs, by keeping the ONNX model and quantization scales constant. It finds that INT8 performance and output consistency vary significantly: CPU dot‑product instructions determine speedup, identical INT8 outputs only occur when integer kernels match, and vendor NPUs require their own quantization pipelines. The authors also show that edge‑NPU latency is dominated by data transfer rather than compute and provide scripts and reports for reproducibility.

By Yuyeong Shin
arXiv Machine Learning
Oct 28

End-to-End MXFP8 Reinforcement Learning on Blackwell: What Matching Buys, When It Pays Off, and How It Fails

The paper presents an end‑to‑end MXFP8 reinforcement learning implementation on Blackwell GPUs, evaluating its benefits and limitations with Qwen3-8B and Qwen3-30B-A3B models. It finds that matching FP8 precision between sampler and learner improves consistency, that speed gains appear only for large micro‑batches, and that several weight‑synchronization correctness conditions can fail silently. The study also introduces seven explicit checks to detect such failures during training.

By Wengee Zhang