← Back to all news
arXiv Machine Learning By Anthony Kostal-Vazquez, Vassil Dimitrov, Branden Butler

Algebraic Integer Quantization for Machine Learning Applications

Read the original on arXiv Machine Learning →

The paper introduces algebraic integer number formats (AINF) as a framework for highly configurable and accurate compressed computation, emphasizing its reduction to purely integer arithmetic for error‑free multiply‑accumulate operations. A matrix multiplication engine leveraging AINF achieves performance close to the most efficient numerical counterparts on modern hardware accelerators. The authors further propose algebraic integer quantization (AIQ) as a lossy compression format for large language model inference, demonstrating competitive accuracy against the industry‑standard MX format at equivalent bit‑per‑parameter budgets, and suggest that higher‑order algebraic integers could yield even greater computational benefits.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 16

Fractional Power-of-Two Quantization for Efficient and Effective Multiplier-Free LLM Inference

The paper introduces Fractional Power-of-Two (FPoT) quantization, a base‑√2 weight grid that enables 4‑bit weight quantization for large language models while keeping a fully integer shift‑add datapath. FPoT combines a fractional PoT grid with uniform integer activations, a calibration‑time grid‑alignment refinement, and a lightweight 1‑adder √2 approximation to reduce gate count compared to a 4‑bit multiplier. On Llama‑3‑8B, FPoT boosts zero‑shot commonsense reasoning by 8.72% over the standard PoT baseline and stays within 0.4% of uniform integer performance, with benefits extending across seven models and three bit‑widths.

By Sunghyun Wee, Geunjae Choi, Hyeonjin Kim, Suyoung Kim, Nojun Kwak
llmsefficiencybenchmarkssafety
More like this →
arXiv AI
Aug 18

FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy

arXiv:2608. 15602v1 Announce Type: cross Abstract: While binary quantization theoretically promises extreme compression and acceleration for Large Language Models (LLMs), existing research often overlooks the necessity of specialized hardware kernels, thus failing to unleash the full acceleration potential due to persistent reliance on expensive floating-point arithmetic or runtime dequantization overheads.

By Qingyao Yang, Runming Yang, He Xiao, Wendong Xu, Junyu Chen, Haobo Liu, Chenchen Ding, Ruihan Hu, Yik-Chung Wu, Ngai Wong
llmsfine-tuningefficiency
More like this →
arXiv Machine Learning
Sep 16

LLM Inference in a Flash!

The paper "LLM Inference in a Flash!" proposes an integer‑only quantization scheme and a dictionary‑based KV cache compression technique to enable large language model inference on compute‑in‑flash (CIF) devices. By eliminating floating‑point operations and reducing KV cache traffic through sparse dictionary coding, the authors achieve minimal accuracy loss while cutting dynamic KV cache traffic by 15× on Llama‑3.1‑8B and Qwen‑2.5‑7B models.

By Sebastian Zhao, Minseo Kim, Coleman Hooper, Luca Manolache, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, Amir Gholami
llmsragnlpefficiency
More like this →