arXiv Computation and Language November 4, 2026 By Dongwei Wang, Jinhee Kim, Seokho Han, Denis Gudovskiy, Yohei Nakata, Tomoyuki Okuno, KhayTze Peong, Kang Eun Jeon, Jong Hwan Ko, Yiran Chen, Huanrui Yang

MoBiQuant: Mixture-of-Bits Quantization for Token-Adaptive Any-Precision LLM

Read the original on arXiv Computation and Language →

MoBiQuant introduces a Mixture-of-Bits quantization framework that allows large language models to be inferred at varying precisions during runtime. It addresses the poor generalization of existing post‑training quantization methods by mitigating precision‑dependent outlier migration through a recursive residual quantization scheme and a token‑aware router that selects the optimal precision per token. Experiments demonstrate that MoBiQuant matches or exceeds single‑precision PTQ performance while providing significant memory savings and up to 1.34× throughput gains over current any‑precision approaches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Jun 16

MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization

arXiv:2606. 15652v1 Announce Type: new Abstract: 4-bit quantization significantly reduces the memory footprint and accelerates the inference of large language models (LLMs).

By Yangjia Hu, Haodong Wang, Zicong Hong, Qianli Liu, Quanxin Shou, Jian Lin, Song Guo, Xiaowei Shen, Xiangjun Huang, Dian Wang, Jian Yang

llms efficiency

arXiv Machine Learning

Jun 8

AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization

arXiv:2605. 08692v2 Announce Type: replace Abstract: Post-training weight-only quantization to 4 bits is widely used to reduce the memory and compute costs of large language model inference.

By Beshr IslamBouli, David Jin

llms efficiency

arXiv Machine Learning

Sep 10

Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

Squeeze10-LLM is a staged mixed‑precision post‑training quantization framework that reduces 16‑bit LLM weights to an average of 1.6 bits per weight by assigning 80% of weights to 1 bit and 20% to 4 bits. It introduces Post‑Binarization Activation Robustness (PBAR), a weight significance metric that considers activation impact, and Full Information Activation Supervision (FIAS), a strategy that preserves activation information to limit error propagation. Experiments on LLaMA and LLaMA2 demonstrate that Squeeze10‑LLM achieves state‑of‑the‑art performance for sub‑2‑bit weight‑only quantization, raising average accuracy from 43% to 56% on six zero‑shot classification tasks.

By Qingcheng Zhu, Yangyang Ren, Linlin Yang, Yanjing Li, Sheng Xu, Haodong Zhu, Juan Zhang, Runqi Wang, Baochang Zhang

llms efficiency benchmarks

arXiv Machine Learning

Jul 3

QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs

arXiv:2602. 10431v4 Announce Type: replace Abstract: Large language models (LLMs) demand substantial computational and memory resources, posing challenges for efficient deployment.

By Kanghyun Noh, Jinheon Choi, Yulhwa Kim

llms fine-tuning efficiency benchmarks

arXiv AI

Jun 6

Channel-Wise Mixed-Precision Quantization for Large Language Models

arXiv:2410. 13056v4 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated remarkable success across a wide range of language tasks, but their deployment on edge devices remains challenging due to the substantial memory requirements imposed by their large parameter sizes.

By Zihan Chen, Bike Xie, Jundong Li, Cong Shen

llms efficiency

arXiv Machine Learning

Jun 3

OASIS: Outlier-Aware LUT-Based GEMM with Dual-Side Quantization for LLM Inference Acceleration

arXiv:2507. 23035v4 Announce Type: replace Abstract: Large language models (LLMs) have demonstrated impressive capabilities across a wide range of applications, but demand substantial memory and compute resources during inference.

By Xueying Wu, Baijun Zhou, Zhihui Gao, Yuzhe Fu, Qilin Zheng, Yintao He, Hai Li

llms efficiency

One email a morning, machine-written

Related stories

MosaicQuant: Inlier-Outlier Disaggregation for Unified 4-Bit LLM Quantization

AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization

Squeeze10-LLM: Squeezing LLMs' Weights by 10 Times via a Staged Mixed-Precision Quantization Method

QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMs

Channel-Wise Mixed-Precision Quantization for Large Language Models

OASIS: Outlier-Aware LUT-Based GEMM with Dual-Side Quantization for LLM Inference Acceleration