arXiv Machine Learning October 3, 2026 By Don Li

Q-MINO: A Minimal-Norm Method for Quantization-Aware Training

Read the original on arXiv Machine Learning →

The paper introduces Q-MINO, a Quantization-Aware Minimal-Norm Optimizer designed to improve training of ultra-low-bit neural networks. Q-MINO uses a temporal bundle method that incorporates gradient consensus, state-drift regularization, and an alignment constraint to produce stabilized, minimum-norm update directions. The authors solve the resulting constrained subproblem with a warm-started Frank–Wolfe procedure and provide theoretical convergence guarantees via a stochastic Lyapunov Kurdyka–Łojasiewicz framework, along with numerical experiments demonstrating its effectiveness across various quantization levels.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Jun 19

CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training

arXiv:2510. 18784v3 Announce Type: replace Abstract: Despite significant work on low-bit quantization-aware training (QAT), there is still an accuracy gap between such techniques and native training.

By Soroush Tabesh, Mher Safaryan, Andrei Panferov, Alexandra Volkova, Dan Alistarh

llms fine-tuning efficiency benchmarks

arXiv AI

Aug 20

Entropy-Constrained Adaptive Stochastic Quantization

arXiv:2608. 18147v1 Announce Type: cross Abstract: Adaptive stochastic quantization (ASQ) is a recently introduced quantization approach that optimizes the Mean Squared Error (MSE) for a given input while preserving unbiasedness.

By Ran Ben Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher, Shay Vargaftik

efficiency

arXiv AI

Jun 15

Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost

arXiv:2602. 03120v2 Announce Type: replace-cross Abstract: Post-Training Quantization (PTQ) is essential for deploying Large Language Models (LLMs) on memory-constrained devices, yet it renders models static and difficult to fine-tune.

By Yinggan Xu, Kajetan Schweighofer, Risto Miikkulainen, Xin Qiu

llms reinforcement-learning fine-tuning efficiency benchmarks

arXiv Machine Learning

Jun 16

Quantization Robustness of Monotone Operator Equilibrium Networks

arXiv:2603. 10562v2 Announce Type: replace-cross Abstract: Monotone operator equilibrium networks are implicit-layer models whose output is the unique equilibrium of a monotone operator, guaranteeing existence, uniqueness, and convergence.

By James Li, Philip H. W. Leong, Thomas Chaffey

efficiency

arXiv Machine Learning

Jun 2

Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms

arXiv:2606. 00289v1 Announce Type: new Abstract: Quantization is a fundamental tool used to compress datasets, neural network weights, and memory usage in a range of computational tasks.

By Nathan White, Krish Singal

efficiency benchmarks

arXiv Machine Learning

Aug 17

QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction

arXiv:2608. 13966v1 Announce Type: new Abstract: As large language model inference shifts toward lower precision, post-training quantization (PTQ) becomes increasingly brittle, making quantization-aware training (QAT) essential for preserving model quality.

By Vincent Counathe, Ben Athiwaratkun, Christopher De Sa, Tianyi Zhang

llms efficiency

One email a morning, machine-written

Related stories

CAGE: Curvature-Aware Gradient Estimation For Accurate Quantization-Aware Training

Entropy-Constrained Adaptive Stochastic Quantization

Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost

Quantization Robustness of Monotone Operator Equilibrium Networks

Inner Product Aware Quantization: Provably Fast, Accurate, and Adaptive Algorithms

QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction