arXiv AI By I Kennedy, T Kennedy

Quantization Error Is Spectrally Flat: A Single Random Probe Is a Calibrated, Data-Free Sensitivity Estimator, with Application to Budget-Targeted Mixed-Precision Quantization

Read the original on arXiv AI →

The Flow has not summarised this story yet — read it at arXiv AI.

arXiv Machine Learning
Oct 8

Quantize by Drift: Label-Free Mixed-Precision Post-Training Quantization for Text Embedders

The paper introduces a label‑free mixed‑precision post‑training quantization technique for text embedders called Quantize by Drift. It uses quantization‑induced representation drift—measured by re‑encoding a corpus after quantizing a module—to estimate module sensitivity without relevance labels. Across five development embedders, drift‑based ordering of mixed‑precision plans correlates strongly with held‑out retrieval quality (macro Spearman 0.911) and outperforms uniform precision under a hard byte budget, though it does not fully replace fine‑grained redistribution strategies.

By Hyojung Han, Jongmin Kim, Seung-Hun Jeon
arXiv Machine Learning
Oct 16

Task-Phase-Conditioned Expert Routing Is Not a Reliable Proxy for Quantization Sensitivity in MoE FFN Tensors

The paper examines whether expert‑routing statistics from a Mixture‑of‑Experts language model can inform layer‑wise mixed‑precision quantization of routed‑expert feed‑forward networks. By building a routing‑capture pipeline for Qwen‑3.6‑35B‑A3B, the authors derive a composite sensitivity score and use coordinate descent to allocate bits per layer under a fixed budget. Experiments across flat, steep, and random allocation regimes show that routing‑guided reallocation does not outperform a uniform baseline, suggesting that routing behavior is an unreliable proxy for true quantization cost.

By Yifan Liu