← Back to all news
arXiv AI By Ning Yang, Bohan Cui, Zhuowen Liu, Yuting Wang

ResCom: Activation Residual Compensation for Large Language Model Quantization

Read the original on arXiv AI →

ResCom introduces a low‑rank correction that leverages activation rounding residuals to compensate for layer‑output errors in low‑bit weight–activation quantization of large language models. The method selects residual directions based on their impact on output reconstruction and computes the correction in closed form, avoiding gradient training or weight updates. Experiments on Llama‑3‑8B with 4‑bit weights and activations show significant gains: perplexity drops from 7.708 to 7.147, MMLU improves by 2.89 percentage points, and five‑shot GSM8K accuracy rises from 23.96 % to 36.16 %.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Oct 12

DualQuant: Dual-scale Calibration-free Weight-and-Activation Quantization for LLMs

DualQuant introduces a dual‑scale, calibration‑free post‑training quantization framework that jointly optimizes weight‑reconstruction MSE and derives per‑channel activation rescalings, enabling effective 4‑bit weight and activation quantization without calibration data. The method employs a non‑differentiable optimization solved via variable splitting and block coordinate descent, supporting a wide range of formats including MX and NVFP4. Experiments on Llama and Qwen models show DualQuant matches or surpasses calibration‑based SmoothQuant and significantly outperforms other calibration‑free baselines, achieving up to 2.61× memory reduction and 2.46× throughput speedup on an RTX 4090.

By Mahsa Salmani, Sayeh Sharify, Hesham Mostafa
llmsefficiency
More like this →