arXiv Machine Learning
Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 23
Fast Matrix Multiplication in fp8: Certified Coefficient Optimization and Measured Error
arXiv:2609.26077v1 Announce Type: new Abstract: A Strassen-type algorithm has many realizations with the same exact product and multiplication count yet different fp8 error because basis changes resh...
arXiv Machine Learning
4d ago
Behavioral Capacity Certificates for Quantized Language Models
arXiv:2609.37887v1 Announce Type: new Abstract: Activation and key-value cache precision change what a quantized language model computes without altering its stored weights. Direct weight-code bounds...
arXiv AI
Jul 13
Signed Symmetric Quantization for Few-Bit Integers
arXiv:2607. 08779v1 Announce Type: cross Abstract: The signed integer alphabet contains one more negative representable value than positive.
arXiv Computation and Language
Sep 11
Structured Transforms for Low-Overhead Quantization of Language Models
The paper revisits Kashin‑decomposition‑based weight quantization for large language models, introducing an improved algorithm that uses a sign‑randomized Discrete Cosine Transform (DCT) instead of a dense random orthogonal matrix. This change reduces per‑iteration cost from ≠(N^2) to ≠(N log N) and, combined with a greedy alternating‑update scheme, guarantees the four‑peak distribution needed for stable 2‑bit clustering while eliminating the need for multi‑restart k‑means. The resulting JAX pipeline, when paired with OPTQ‑style error compensation and QuIP‑style incoherence preprocessing, competes with state‑of‑the‑art quantization methods on OPT, Llama‑2, and Pythia at 4‑bit per channel, and remains numerically stable under stress configurations that cause other methods to diverge.
arXiv AI
Jun 8
P-Cast Precision in FP8 Attention: Sink-Induced Collapse and the Optimality of S=2^8
arXiv:2606. 06521v1 Announce Type: cross Abstract: FP8 (E4M3) acceleration for attention computation offers significant throughput gains, but the 3-bit mantissa introduces precision challenges when the softmax probability matrix P is cast to FP8 before the P*V matrix multiplication.
arXiv AI
Jun 16
FP8 is All You Need (Part 1): Debunking Hardware FP64 as the HPC Holy Grail (June 13th version)
arXiv:2606. 06510v2 Announce Type: replace-cross Abstract: Conventional HPC holds that native hardware FP64 is the irreducible foundation of scientific computing.