arXiv Machine Learning October 12, 2026 By Tianqi Chen, Zhe Li, Weixiang Xu, Zeyu Zhu, Dong Li, Lu Tian, Emad Barsoum, Peisong Wang, Jian Cheng

TernaryLLM: Ternarized Large Language Model

Read the original on arXiv Machine Learning →

TernaryLLM introduces Dual Learnable Ternarization (DLT) to allow both scales and shifts to be learned, addressing asymmetric outliers and non‑zero means in weights. It also proposes Outlier‑Friendly Feature Knowledge Distillation (OFF), which maximizes mutual information between ternarized and floating‑point features using cosine similarity to recover lost information. Experiments show TernaryLLM outperforms prior low‑bit quantization methods on text generation and zero‑shot benchmarks, notably improving LLaMA‑3’s perplexity and accuracy.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

10h ago

A Comprehensive Evaluation on Quantization Techniques for Large Language Models

arXiv:2507.17417v4 Announce Type: replace Abstract: For large language models (LLMs), post-training quantization (PTQ) significantly reduces memory and computational overhead. Despite rapid advanceme...

By Yutong Liu, Cairong Zhao, Guosheng Hu

llms efficiency benchmarks

Hugging Face Trending Papers

Jun 11

TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization

Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment. Ternarization has emerged as a promising compression technique, offering significant reductions in model size and inference complexity.

llms efficiency

arXiv AI

Jun 12

TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization

arXiv:2606. 13054v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit exceptional general language processing capabilities, but their memory and compute costs hinder deployment.

By Zhixiong Zhao, Zukang Xu, Zhixuan Chen, Xing Hu, Zhe Jiang, Dawei Yang

llms efficiency

arXiv AI

6d ago

RoleQ: Role-Aligned Scales for Converting Language Models to Ternary

arXiv:2610. 21808v1 Announce Type: new Abstract: Weight quantization below four bits shrinks memory and bandwidth at a perplexity that looks acceptable, but often at the cost of factual knowledge and task accuracy.

By Johannes Knittel, Alec Helbling, Hanspeter Pfister

llms fine-tuning efficiency benchmarks

arXiv Machine Learning

10h ago

InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization

InfoQuant is a train‑free method that reshapes activation distributions to be more compatible with low‑bit quantizers. It uses Peak Suppression Orthogonal Transformation (PSOT) and adaptive outlier‑token selection to reduce numerical range while maintaining dispersion, thereby lowering quantization error. Across several large language model families, InfoQuant outperforms existing post‑training quantization and end‑to‑end training baselines, preserving 97% of floating‑point accuracy on average and narrowing the LLaMA‑2 13B performance gap by 42%.

By Ke Li, Dong An, Xiaoling Zang, Can Ye, Liang Xie, Qibo Qiu, Chen Shen, Xiaofei He, Wenxiao Wang

llms efficiency

arXiv AI

Jun 26

CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs

arXiv:2606. 26650v1 Announce Type: cross Abstract: In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs.

By Shigeng Wang, Chao Li, Yangyuxuan Kang, Jiawei Fan, Anbang Yao

llms efficiency benchmarks

One email a morning, machine-written

Related stories

A Comprehensive Evaluation on Quantization Techniques for Large Language Models

TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization

TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization

RoleQ: Role-Aligned Scales for Converting Language Models to Ternary

InfoQuant: Shaping Activation Distributions for Low-Bit LLM Quantization

CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs