← Back to all news
arXiv Machine Learning

Algebraic Integer Quantization for Machine Learning Applications

The paper introduces algebraic integer number formats (AINF) as a framework for highly configurable and accurate compressed computation, emphasizing its reduction to purely integer arithmetic for error‑free multiply‑accumulate operations. A matrix multiplication engine leveraging AINF achieves performance close to the most efficient numerical counterparts on modern hardware accelerators. The authors further propose algebraic integer quantization (AIQ) as a lossy compression format for large language model inference, demonstrating competitive accuracy against the industry‑standard MX format at equivalent bit‑per‑parameter budgets, and suggest that higher‑order algebraic integers could yield even greater computational benefits.

Related stories

arXiv AI
Oct 16

Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference

The paper introduces Fractional Power-of-Two (FPoT) quantization, a base‑√2 weight grid that enables 4‑bit weight quantization for large language models while keeping a fully integer shift‑add datapath. FPoT combines a fractional PoT grid with uniform integer activations, a calibration‑time grid‑alignment refinement, and a lightweight 1‑adder √2 approximation to reduce gate count compared to a 4‑bit multiplier. On Llama‑3‑8B, FPoT boosts zero‑shot commonsense reasoning by 8.72% over the standard PoT baseline and stays within 0.4% of uniform integer performance, with benefits extending across seven models and three bit‑widths.

By Sunghyun Wee, Geunjae Choi, Hyeonjin Kim, Suyoung Kim, Nojun Kwak
llmsefficiencybenchmarkssafety
More like this →
arXiv AI
Aug 18

FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy

arXiv:2608. 15602v1 Announce Type: cross Abstract: While binary quantization theoretically promises extreme compression and acceleration for Large Language Models (LLMs), existing research often overlooks the necessity of specialized hardware kernels, thus failing to unleash the full acceleration potential due to persistent reliance on expensive floating-point arithmetic or runtime dequantization overheads.

By Qingyao Yang, Runming Yang, He Xiao, Wendong Xu, Junyu Chen, Haobo Liu, Chenchen Ding, Ruihan Hu, Yik-Chung Wu, Ngai Wong
llmsfine-tuningefficiency
More like this →
arXiv Machine Learning
Sep 16

LLM Inference in a Flash!

The paper "LLM Inference in a Flash!" proposes an integer‑only quantization scheme and a dictionary‑based KV cache compression technique to enable large language model inference on compute‑in‑flash (CIF) devices. By eliminating floating‑point operations and reducing KV cache traffic through sparse dictionary coding, the authors achieve minimal accuracy loss while cutting dynamic KV cache traffic by 15× on Llama‑3.1‑8B and Qwen‑2.5‑7B models.

By Sebastian Zhao, Minseo Kim, Coleman Hooper, Luca Manolache, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, Amir Gholami
llmsragnlpefficiency
More like this →
arXiv AI
2d ago

"Close enough?" An investigation into multiplication errors in LLMs

The paper investigates why large language models (LLMs) struggle with multi‑digit multiplication. By testing several small base LLMs on 16,991 multiplication problems, the authors find that errors do not follow the previously observed noisy‑quantisation pattern seen in addition. Instead, small Qwen models exhibit a new error regime: lower‑order column mistakes tend to preserve parity, while higher‑order digit errors are mainly off‑by‑one carry failures, indicating distinct internal representations for arithmetic in transformers.

By William Bolton, Bezawit Tilaye, Huw Day
llms
More like this →