arXiv AI August 18, 2026 By Qingyao Yang, Runming Yang, He Xiao, Wendong Xu, Junyu Chen, Haobo Liu, Chenchen Ding, Ruihan Hu, Yik-Chung Wu, Ngai Wong

FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy

Read the original on arXiv AI →

arXiv:2608. 15602v1 Announce Type: cross Abstract: While binary quantization theoretically promises extreme compression and acceleration for Large Language Models (LLMs), existing research often overlooks the necessity of specialized hardware kernels, thus failing to unleash the full acceleration potential due to persistent reliance on expensive floating-point arithmetic or runtime dequantization overheads.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Jun 16

MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization

arXiv:2606. 15652v1 Announce Type: new Abstract: 4-bit quantization significantly reduces the memory footprint and accelerates the inference of large language models (LLMs).

By Yangjia Hu, Haodong Wang, Zicong Hong, Qianli Liu, Quanxin Shou, Jian Lin, Song Guo, Xiaowei Shen, Xiangjun Huang, Dian Wang, Jian Yang

llms efficiency

arXiv AI

Jul 13

Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices

arXiv:2607. 08786v1 Announce Type: cross Abstract: With the growing deployment of large language models (LLMs), LLM inference cost has become a key challenge.

By Tao Lu, Haoyu Wang, Zonghui Wang, Keshen Xiang, Jiaheng Zhang, Wenzhi Chen

llms efficiency

arXiv AI

Jul 7

ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs

arXiv:2601. 07475v2 Announce Type: replace-cross Abstract: The emergence of fine-grained numerical formats like NVFP4 presents new opportunities for efficient Large Language Model (LLM) inference.

By Haoqian Meng, Yilun Luo, Yafei Zhao, Wenyuan Liu, Peng Zhang, Xindian Ma

llms efficiency benchmarks

arXiv Machine Learning

Jun 16

NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models

arXiv:2602. 06694v3 Announce Type: replace Abstract: Weight-only quantization has become a standard approach for efficiently serving large language models (LLMs).

By Hyochan Chong, Dongkyu Kim, Changdong Kim, Minseop Choi

llms efficiency

arXiv AI

Jun 16

RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs

arXiv:2602. 05367v3 Announce Type: replace Abstract: Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a critical trade-off between low-bit efficiency and performance.

By Youngcheon You, Banseok Lee, Minseop Choi, Seonyoung Kim, Hyochan Chong, Changdong Kim, Youngmin Kim, Dongkyu Kim

llms efficiency benchmarks

arXiv Machine Learning

Jul 27

Unified Static-Dynamic Pruning for Efficient LLM Inference

arXiv:2607. 21985v1 Announce Type: cross Abstract: The increasing deployment of large language models (LLMs) has magnified the computational and memory bottlenecks of autoregressive decoding, where low compute intensity and bandwidth-bound kernels dominate inference cost.

By Jinhyeok Kim, Yejoon Lee, Jaeyoung Do

llms efficiency

Related stories

MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization

Accelerating GPU Inference of Large Language Models with Moderately Unstructured Sparse Weight Matrices

ARCQuant: Boosting NVFP4 Quantization with Augmented Residual Channels for LLMs

NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models

RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs

Unified Static-Dynamic Pruning for Efficient LLM Inference