MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization
Read the original on Hugging Face Trending Papers →MultiAlloc is a profiling‑guided framework that allocates limited quantization resources between layer‑wise precision enhancement and token‑wise preservation for KV cache quantization in large language models. By using lightweight offline profiling to assess layer sensitivity and token importance, it automatically derives an effective precision allocation strategy that satisfies a fixed memory budget without online search. Experiments on reasoning and long‑context benchmarks show that MultiAlloc consistently outperforms existing layer‑wise, token‑wise, and multi‑granularity quantization methods across models such as Mistral, Llama, DeepSeek‑Qwen, and QwQ.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.