CoS-VQ: Coordinated Shared-Scale Vector Quantization for Large Language Models

CoS‑VQ is a new vector‑quantization technique for large language models that keeps codewords low‑dimensional while jointly optimizing their hard indices and a shared scale across positions. By collecting candidates at multiple probe scales and using a tangent separable upper bound, it reduces candidate comparisons to a small set of representative intervals, enabling efficient joint index–scale updates without exhaustive enumeration. Experiments on Qwen3.8‑27B show that CoS‑VQ outperforms existing methods such as QTIP and LLVQ on language‑modeling, coding, and reasoning benchmarks, and ablations demonstrate that joint updates yield larger gains than scale‑only refinement.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

• llms
• efficiency
• benchmarks