← Back to all news
arXiv Machine Learning By Neha Verma, Sungwon Kim, Kenton Murray, Kevin Duh

VFold: Symmetry-Aware Cross-Layer Value Cache Compression

Read the original on arXiv Machine Learning →

The paper introduces VFold, a symmetry‑aware strategy for merging value caches across layers in large language models to reduce memory usage during decoding. It avoids the performance penalties and architectural changes typical of other compression methods, and can be combined with high‑ratio quantization or key cache pruning to achieve higher compression ratios. The authors demonstrate that a significant portion of the value cache is underutilized, providing a straightforward path to scaling context windows within memory limits.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

Hugging Face Trending Papers
Oct 8

VFold: Symmetry-Aware Cross-Layer Value Cache Compression

VFold introduces a symmetry‑aware value cache merging strategy that compresses the key‑value cache used during Large Language Model decoding. By exploiting inter‑layer cache similarities, it reduces memory usage without causing performance loss or requiring architectural changes. The method can be combined with other compression techniques, such as high‑ratio quantization or key cache pruning, to achieve higher compression ratios with minimal extra cost.

llmsefficiency
More like this →
Hugging Face Trending Papers
Oct 13

MultiAlloc: Budget-Aware Precision Allocation for KV Cache Quantization

MultiAlloc is a profiling‑guided framework that allocates limited quantization resources between layer‑wise precision enhancement and token‑wise preservation for KV cache quantization in large language models. By using lightweight offline profiling to assess layer sensitivity and token importance, it automatically derives an effective precision allocation strategy that satisfies a fixed memory budget without online search. Experiments on reasoning and long‑context benchmarks show that MultiAlloc consistently outperforms existing layer‑wise, token‑wise, and multi‑granularity quantization methods across models such as Mistral, Llama, DeepSeek‑Qwen, and QwQ.

llmsefficiencybenchmarks
More like this →