arXiv AI September 30, 2026 By Ran Ben Basat, Michael Mitzenmacher, Shay Vargaftik

TORQUE: Optimizing What (not) to Quantize Before and After Rotation

The paper introduces TORQUE, a framework that enhances quantization by jointly optimizing which coordinates to keep at high precision before and after applying uniform random rotations, all within a fixed bit budget. By preserving large input coordinates before rotation and the largest-magnitude coordinates after rotation, TORQUE reduces quantization error and allows efficient use of offline-optimized codebooks. The authors provide an error upper bound, prove that top‑k pre‑rotation retention is optimal for each k, and demonstrate improved accuracy‑storage tradeoffs in Gaussian models and practical tasks such as nearest‑neighbor retrieval, KV‑cache compression, and activation compression.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

efficiency

arXiv Computer Vision

Sep 22

SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models

SPHQuant introduces a rotation‑free spherical weight‑only quantization framework for Vision‑Language Models, decomposing 8‑dimensional weight vectors into sign, radius, and a positive unit direction. By isolating outlier magnitudes in the radius and allocating extra precision there, it mitigates accuracy loss at extreme low bit‑widths. The method also employs a compact positive‑direction codebook with angular fine‑tuning and a hardware‑friendly GEMV kernel, achieving state‑of‑the‑art performance while boosting decode throughput by 30.3% on RTX A6000 compared to QTIP.

By Kewei Zhang, Zheng Chen, Haotong Qin, Yulun Zhang

llms efficiency multimodal benchmarks

Hugging Face Trending Papers

Sep 21

SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models

Recent foundation models are moving toward native multimodal Vision-Language Models (VLMs), making VLMs a central form of next-generation foundation models. However, their large language backbones mak...

llms efficiency multimodal benchmarks

arXiv Machine Learning

Sep 1

RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search

arXiv:2608.30384v1 Announce Type: new Abstract: By introducing RSLM (Rotated Scaled Lloyd-Max), a family of training-free vector quantization codecs compressing embeddings to 1--4 bits per dimension,...

By Rastislav Lenhardt, Teodora Dobos, Thomas Vecchiato, Jiri Isa, Igor Ginzburg

rag efficiency benchmarks safety

arXiv Machine Learning

Jun 8

PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration

arXiv:2502. 00527v2 Announce Type: replace Abstract: The KV cache in large language models is a dominant factor in memory usage, limiting their broader applicability.

By Songhao Wu, Ang Lv, Xiao Feng, Yufei Zhang, Xun Zhang, Guojun Yin, Wei Lin, Rui Yan

llms rag efficiency

arXiv AI

Jul 16

NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache

arXiv:2505. 18231v3 Announce Type: replace-cross Abstract: Large Language Model (LLM) inference is typically memory-intensive, especially when processing large batch sizes and long sequences, due to the large size of key-value (KV) cache.

By Donghyun Son, Euntae Choi, Sungjoo Yoo

llms efficiency safety

arXiv AI

Aug 6

Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms

arXiv:2608. 04074v1 Announce Type: cross Abstract: Long-context LLM decoding reads the key-value (KV) cache at every step.

By Samuel Fern\'andez-Mendui\~na, Amir Ziashahabi, Eduardo Pavez, Antonio Ortega, Salman Avestimehr

llms efficiency

One email a morning, machine-written

Related stories

SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models

SPHQuant: Efficient extreme low bit weight quantization for Vision-Language Models

RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search

PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration

NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache

Spend Bits Where Queries Look: KV Cache Vector Quantization with Attention-Preserving Transforms