← Back to all news
arXiv AI By Yuechuan Chen, Yanda Guo, Xiaoxi Liu, Weilin Ruan, Huanbo Xu, Ruyi Wang, Wei Zhu

ChampionKV: Head-Wise Adaptive Budget Allocation for KV Cache Compression

Read the original on arXiv AI →

ChampionKV introduces an input‑adaptive, head‑wise budget allocator for KV cache compression, using the maximum aggregated observation‑window attention score as a lightweight proxy for attention concentration. It assigns larger budgets to heads with dispersed attention and smaller budgets to concentrated heads, and incorporates an abnormal‑head mask and floor‑budget mechanism to enhance robustness. When integrated with existing compressors like SnapKV and PyramidKV, ChampionKV consistently improves performance on LongBench and RULER, especially on Llama3.1‑70B‑Instruct under tight budgets, while maintaining competitive results on smaller models.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computation and Language
Aug 25

Beyond Sparse Weights: When Is Attention Compressible?

The paper investigates when attention maps can be compressed beyond simple sparsity, arguing that large weights alone do not guarantee compressibility. It introduces metrics such as global score gaps and weighted sums of omitted values to determine token retention for a target mass, and presents a retrieval–aggregation model to predict the impact of truncation. Based on these insights, the authors propose CertKV, a training‑free compressor that allocates a tail‑summary slot per head and distributes remaining slots according to value dispersion, achieving strong performance across several benchmarks.

By Chiwun Yang, Xiaoyu Li
llms
More like this →
arXiv AI
Jun 9

From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs

arXiv:2606. 09508v1 Announce Type: new Abstract: Existing sparse attention and KV cache compression methods for long-context LLM inference typically apply fixed sparsity patterns or uniform budgets across all attention heads, overlooking the substantial variation in attention behavior among heads and contexts.

By Zhanchao Xu, Haoyang Li, Qingfa Xiao, Fei Teng, Chen Jason Zhang, Lei Chen, Qing Li
llmsefficiency
More like this →
arXiv AI
Sep 3

HeadWiseKV: Budgeted Per-Head Cache Residency for Hybrid Long-Context Language Models

HeadWiseKV is a training‑free framework that compresses the residual global key–value caches of hybrid long‑context language models by assigning each physical KV head a static, multilevel history window. It formulates cache allocation as a restricted operational rate–distortion problem and uses the SeqCalib algorithm to generate per‑head residency policies that account for interactions across layers. In evaluations on four hybrid models, HeadWiseKV preserves near‑full‑KV quality while reducing peak device memory usage by 8.59% at a 112K context length and extending the largest verified context from 114K to 161K.

By Renjie Xie, Juncheng Yang, Aoting Hu, Mingxi Zhang, Liyao Wu, Zheheng Hong, Wei Xu
llmsefficiency
More like this →