arXiv AI

XOR-Trellis: Ultra-Low-Complexity Dequantization and Curvature-Aware Hadamard-Free LLM Quantization

arXiv AI

Jun 10

LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization

arXiv:2606. 10531v1 Announce Type: cross Abstract: Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs).

By Haoyu Wang, Xingyu Yu, Haiyan Zhao, Fengxiang Wang, Xu Han

llms efficiency benchmarks

arXiv AI

Jun 4

LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection

arXiv:2606. 04050v1 Announce Type: cross Abstract: Existing quantization methods are fundamentally limited by rigid, integer-based bit-widths (e.

By Liulu He, XuanAng Liu, Juntao Liu, Taolue Feng, Ting Lu, Chunsheng Gan, Zhiyv Peng, Yuan Du, Huanrui Yang, Yijiang Liu, Li Du

llms efficiency benchmarks

arXiv AI

Jun 6

LLMCodec: Adapting Video Codecs for Efficient Weight Compression of Large Language Models

arXiv:2606. 05861v1 Announce Type: cross Abstract: The rapid development of large language models(LLMs) has led to remarkable advances in natural language processing.

By Rui Wang, Yan Zhao, Li Song, Zhengxue Cheng

llms nlp fine-tuning efficiency

arXiv AI

Jun 12

TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization

arXiv:2606. 13054v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment.

By Zhixiong Zhao, Zukang Xu, Zhixuan Chen, Xing Hu, Zhe Jiang, Dawei Yang

llms efficiency

Hugging Face Trending Papers

Jun 11

TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization

Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment. Ternarization has emerged as a promising compression technique, offering significant reductions in model size and inference complexity.

llms efficiency

arXiv Machine Learning

Jun 16

NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models

arXiv:2602. 06694v3 Announce Type: replace Abstract: Weight-only quantization has become a standard approach for efficiently serving large language models (LLMs).

By Hyochan Chong, Dongkyu Kim, Changdong Kim, Minseop Choi

llms efficiency

arXiv AI

Jun 9

Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression

arXiv:2606. 07819v1 Announce Type: new Abstract: Recently, the efficiency of Large Language Models (LLMs) deployment has become a critical concern in practical applications.

By Hoang-Loc La, Truong-Thanh Le, Amir Taherkordi, Phuong Hoai Ha

llms efficiency benchmarks

arXiv Machine Learning

Jun 16

MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization

arXiv:2606. 15652v1 Announce Type: new Abstract: 4-bit quantization significantly reduces the memory footprint and accelerates the inference of large language models (LLMs).

By Yangjia Hu, Haodong Wang, Zicong Hong, Qianli Liu, Quanxin Shou, Jian Lin, Song Guo, Xiaowei Shen, Xiangjun Huang, Dian Wang, Jian Yang

llms efficiency

arXiv Machine Learning

Jun 8

AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization

arXiv:2605. 08692v2 Announce Type: replace Abstract: Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference.

By Beshr IslamBouli, David Jin

llms efficiency

arXiv AI

Jun 18

HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning

arXiv:2601. 21626v2 Announce Type: replace-cross Abstract: Post Training Quantization (PTQ), a mainstream model compression technique, often leads to the paradoxical 'low error, high loss' phenomenon because it focuses solely on minimizing quantization error.

By Jinhao Zhang, Yunquan Zhang, Zicheng yan, Boyang Zhang, Jun Sun, Daning Cheng

llms efficiency

arXiv Machine Learning

Sep 1

ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Models

arXiv:2605.11222v2 Announce Type: replace Abstract: Quantization is an effective strategy to reduce the storage and computation footprint of large language models (LLMs). Post-training quantization (...

By Ryan Lucas, Mehdi Makni, Xiang Meng, Adam Deng, Rahul Mazumder

llms efficiency

arXiv AI

Jun 16

RaBiT: Residual-Aware Binarization Training for Accurate and Efficient LLMs

arXiv:2602. 05367v3 Announce Type: replace Abstract: Efficient deployment of large language models (LLMs) requires extreme quantization, forcing a critical trade-off between low-bit efficiency and performance.

By Youngcheon You, Banseok Lee, Minseop Choi, Seonyoung Kim, Hyochan Chong, Changdong Kim, Youngmin Kim, Dongkyu Kim

llms efficiency benchmarks

Related stories