← Back to all news
arXiv AI By Artur Zagitov, Gleb Molodtsov, Aleksandr Beznosikov

HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization

Read the original on arXiv AI →

HARP (Hadamard‑Preconditioned Adaptive Rotation Processor) is a learnable, structured two‑sided orthogonal processor that replaces fixed randomized Hadamard transforms in post‑training quantization of large language models. By representing rotations as sparse butterfly‑like block‑orthogonal stages and supporting mixed‑radix schedules, HARP adapts the quantization basis to each layer and calibration distribution while maintaining full‑precision equivalence. Across 2–4‑bit settings on Llama models from 1B to 70B, HARP consistently improves perplexity, delivers the strongest zero‑shot gains at 2 bits, and preserves deployment efficiency—achieving 128 tokens per second on Llama 2 7B at 2 bits, roughly 90% of RHT throughput and over twice the speed of FP16.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 15

PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

PrismQuant introduces a rotation framework that aligns the leading activation eigenspace with the constant group subspace of asymmetric grouped INT4 quantizers, optimizing the alignment via Ky Fan trace maximization and providing a closed‑form solution. The method employs compact Householder transformations for efficient, gradient‑free construction and applies a predictive range law to relate unaligned activation energy and group size to quantization variation. Experiments on large language models up to 70B parameters show PrismQuant achieving state‑of‑the‑art perplexity and accuracy, while delivering significant speedups and memory reductions in deployment scenarios.

By Yanlong Chen, Yining Chen, Song Zhang, Amirhossein Habibian, Yawei Li
llmsefficiencysafety
More like this →
arXiv Machine Learning
Aug 11

RotaryQuant: Fitting 120B MoE Models on Consumer Hardware via Fused Compressed-Space Attention

arXiv:2608. 08081v1 Announce Type: cross Abstract: Large mixture-of-experts (MoE) language models with 26--120 billion parameters exceed the memory capacity of consumer devices through three simultaneous pressures: resident weight matrices, key-value (KV) cache state that grows linearly with context, and dozens of expert sublayers that must be paged on demand.

By Anthony. Lui, Mohamed. Elsaied, N. P. Savani
llmsefficiency
More like this →