We introduce ExTernD (Expanded-rank Ternary Decomposition), a post-training factorization of each LLM weight matrix $A \in \mathbb{R}^{m \times n}$ into $A \approx B \mathrm{diag}(D) C$ with ternary factors $B \in \{-1,0,+1\}^{m \times k}$, $C \in \{-1,0,+1\}^{k \times n}$ and a real scale vector $D \in \mathbb{R}^k$. The inner rank $k = μ\min(m,n)$ is deliberately expanded beyond full rank ($μ> 1$), so that components past full rank correct the quantization error of earlier ones.
ShamAN-Q is a sub‑1‑bit post‑training quantization technique that builds on NanoQuant by replacing its diagonal reconstruction geometry with a dense curvature metric inspired by the Shampoo optimizer. For each linear weight, it fits a Kronecker product to the empirical Fisher information matrix of a small calibration set via Kullback–Leibler minimization, yielding a Mahalanobis reconstruction loss. The method updates continuous ADMM steps to Sylvester equations while keeping the discrete projection and deployment format unchanged, and it redistributes uniform rank across layers, achieving lower perplexity on Qwen3‑Base at roughly 1 bpw and matching or improving zero‑shot accuracy on the Eleuther LM Evaluation Harness.
By Jonathan Mei, Sang Hyub Kim, Oliver Knitter, Chi Chen, Martin Roetteler
arXiv:2609.00224v1 Announce Type: cross
Abstract: Weight-only post-training quantization (PTQ) can alleviate the computational burden of serving large language models (LLMs) at scale. However, existi...
By Yipin Guo, Arun M George, Jie Fu, Tareq Mahmoud, Sixue Xing, Siddharth Joshi
arXiv:2608.28150v2 Announce Type: replace
Abstract: How much matrix rank is required to preserve every bounded value output of normalized softmax attention? We study the unrestricted maximum-row-\(\e...
By Yuhe Sui, Jianing Zhang, Yingzhi Tang
arXiv:2605.11222v2 Announce Type: replace
Abstract: Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (...
By Ryan Lucas, Mehdi Makni, Xiang Meng, Adam Deng, Rahul Mazumder
The paper introduces a ternary multiplicative adaptation technique that enables fine‑tuning of ternary transformers without dequantization. By representing discrete weight updates as a low‑rank Kronecker factorization of two small ternary matrices applied element‑wise, the method preserves the ternary domain and allows direct merging of adaptation weights. Experiments on six language and vision models, including ternarized LLaMA‑3 and ViT‑B/16, show that the approach recovers most of the performance lost to quantization and outperforms existing low‑bit and ternary baselines.
By Alexandru-Dragos Manolache, Yunqiang Li, Jan van Gemert
The paper reports a large‑scale post‑training ternarisation of the Qwen3 language model, extending a conversion pipeline from the 4B to the 8B variant. Using KOTMS rotation, E2M‑ATQ adaptive ternarisation, and GPTQ‑style error compensation, the authors achieve a 1.361× perplexity ratio across three corpora and retain 78.5% of the FP16 accuracy on zero‑shot tasks, with the 8B model outperforming the 4B by 8.9 percentage points. The study also demonstrates lossless lattice‑aware packing, producing an 8.24 GiB checkpoint that preserves perplexity, and shows that direct packed execution can reach 15.52 tokens/s in 7.35 GiB, though packed GEMV remains slower than FP16 cuBLAS.
By Anirudh Malik, M Sparsh Mehra, Poojith Devan
arXiv:2607. 18745v1 Announce Type: new Abstract: We study low-precision computation of C=AB with both factors quantized.
By Piyush Sao, Narasinga Miniskar, Pedro Valero-Lara, Keita Teranishi, Sudip Seal
arXiv:2512. 00956v3 Announce Type: replace Abstract: Quantizing LLM weights and activations is a standard approach for efficient deployment, but a few extreme outliers can stretch the dynamic range and amplify low-bit quantization errors.
By Jiale Chen, Vage Egiazarian, Roberto L. Castro, Torsten Hoefler, Dan Alistarh
The paper investigates whether sink-aware attention head selection remains valid after 4‑bit NF4 weight‑only post‑training quantization. Using Sink Topology Consistency metrics, it finds that global rank preservation stays high across Qwen2.5 and Llama‑3.2 models, yet top‑k head overlap drops to 61–79% and layer‑specific sink‑mass shifts can be substantial. The study also shows that cross‑domain calibration degrades more than within‑domain precision and that recalibration with a small number of samples can recover most of the stability, though full‑map stability may require updating more layers.
By Kuanlin Chen, Chen-Wei Kuo, Cheng-En Ou
arXiv:2609.15838v1 Announce Type: cross
Abstract: Per-matrix singular value decomposition (SVD) truncation is Eckart-Young optimal in the whitened Frobenius norm, but errors from independently compre...
By Huicheng Zhang, Xiyao Feng, Ze-Tong Li, Chengkai Zhu, Xiao Shi, Xiwei Pan, Jinguo Liu, Ge Bai, Xin Wang
arXiv:2608. 14149v1 Announce Type: new Abstract: Recent training-free post-training quantization methods restore model accuracy through closed-form residual compensation.
By Lin-Fa Lee, Yi-Yu Chang, Kuo-Hei Yeh