arXiv Machine Learning October 23, 2026 By Faraz Tahmasebi, Haocheng Xu, Andrew Trauger, Yomal Perera, Zhenyu Tang, Nael Chowdhury, Sitao Huang

FIGFLEX: A Mixed-Precision FP-INT GEMM Accelerator Co-Designed with Low-Bit LLM Quantization

Read the original on arXiv Machine Learning →

FIGFLEX is a mixed‑precision accelerator co‑designed with a low‑bit LLM quantizer that performs floating‑point‑integer (FP‑INT) GEMM. It introduces a shift‑add, radix‑4 reformulation of FP‑INT multiplication that fuses multiply‑accumulate into a precision‑flexible partial‑product reduction, and a novel INT3 optimization that reclaims idle datapath leaves. The accompanying mixed‑precision quantizer assigns channel‑group bit‑widths based on GPTQ error, using power‑of‑two column scaling to keep activations out of the FP16 subnormal range, enabling the quantized model to run on FIGFLEX with the same accuracy as on a GPU while achieving higher throughput per area on 3‑bit mixed‑precision workloads.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Hugging Face Trending Papers

Oct 21

FIGFLEX: A Mixed-Precision FP-INT GEMM Accelerator Co-Designed with Low-Bit LLM Quantization

Weight-only quantization keeps LLM activations in floating point and stores weights as low-bit integers, turning the dominant workload into floating-point-integer (FP-INT) general matrix multiplicatio...

llms fine-tuning efficiency

arXiv AI

Oct 2

ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator

ShatterQuant is a hardware-software co-designed framework that enables mixed-precision quantization within individual tensors by assigning different bit-widths to blocks of a weight projection. It couples precision granularity with processing element configuration, allowing each precision to determine an effective block height. The framework includes a hardware-aware post-training method based on block-level standard deviation and weight sensitivity, a ShatterQuant Transformer Accelerator supporting 1/2/4/8-bit weight precision, precision-dependent PE configuration, block rescaling, and integrated softmax and piecewise-linear nonlinearities, and an evaluation showing 1.5 TOPS, 760 GOPS/$mm^2$ area efficiency, and 2.8 TOPS/W energy efficiency on a TSMC 16nm PDK implementation.

By Mikolaj Walczak, Edward Humes, Chao Fang, Marian Verhelst, Tinoosh Mohsenin

llms efficiency benchmarks

arXiv Machine Learning

1d ago

StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference

arXiv:2611.08363v1 Announce Type: new Abstract: Quantization reduces the memory and bandwidth demands of large language model (LLM) inference but can degrade accuracy; offloading higher-precision dat...

By Jake Hyun, Ziyu Zhao, Jae-sun Seo, Mohamed S. Abdelfattah

llms efficiency

arXiv AI

Oct 15

SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usability

arXiv:2610.14745v1 Announce Type: new Abstract: Extreme low-bit weight-only quantization is an effective approach to reducing the deployment costs of large language models. Below 3 bits, scalar quant...

By Xianglong Yan, Zenan Wu, Kewei Zhang, Kaicheng Yang, Yulun Zhang

llms agents efficiency benchmarks

arXiv Machine Learning

Jun 4

SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Models

arXiv:2602. 01027v2 Announce Type: replace Abstract: Mixed-precision quantization is a promising approach for compressing large language models under tight memory budgets.

By Xin Nie, Haicheng Zhang, Liang Dong, Beining Feng, Jinhong Weng, Guiling Sun

llms efficiency benchmarks

arXiv AI

Oct 7

AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation

arXiv:2610.07457v1 Announce Type: cross Abstract: Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GP...

By Hanzhi Zhang, Qiao Zhang, Qinglei Cao, Heng Fan, Yan Huang, Kewei Sha, Yunhe Feng

llms efficiency

One email a morning, machine-written

Related stories

FIGFLEX: A Mixed-Precision FP-INT GEMM Accelerator Co-Designed with Low-Bit LLM Quantization

ShatterQuant: Breaking Uniform Precision with Block-Wise Mixed-Precision on a Systolic Transformer Hardware Accelerator

StrataFlux: Memory-Hierarchy-Native Low-Bit LLM Inference

SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usability

SFMP: Fine-Grained, Hardware-Friendly and Search-Free Mixed-Precision Quantization for Large Language Models

AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation