arXiv Computation and Language September 1, 2026 By Junjie Peng, You Wu, Haoyi Wu, Jialong Han, Xiaohua Xie, Kewei Tu, Jianhuang Lai

GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs

Read the original on arXiv Computation and Language →

The paper introduces GRKV, a training‑free method for compressing the key‑value cache in long‑context large language models. GRKV uses ridge‑regression to redistribute information from evicted tokens to retained ones, aiming to minimize the difference between compressed‑cache and full‑cache attention outputs. Experiments on LongBench and RULER show that GRKV improves overall performance with minimal overhead compared to other merging methods.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Oct 2

PatchKV: Weight-Space Compensation of KV Cache

arXiv:2609.39329v1 Announce Type: new Abstract: Long-context inference with Large Language Models (LLMs) is bottlenecked by the linearly growing memory of the key-value (KV) cache. Existing compressi...

By Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi, Donggyun Kim, Seunghoon Hong

llms efficiency benchmarks

Hugging Face Trending Papers

Jun 23

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware. Existing KV cache eviction methods typically apply heuristic token scoring over all heads in GQA-based LLMs.

llms efficiency

arXiv AI

Jun 24

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

arXiv:2606. 24467v1 Announce Type: new Abstract: Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware.

By Xiaolin Lin, Jingcun Wang, Olga Kondrateva, Yiyu Shi, Bing Li, Grace Li Zhang

llms efficiency

arXiv AI

Jul 21

SelKV: Selective KV Cache Merging with Per-Token Merge-or-Drop and Attention Compensation

arXiv:2607. 16213v1 Announce Type: new Abstract: Large Language Models (LLMs) generate text autoregressively, relying on a key-value (KV) cache whose memory footprint grows linearly with context length, creating a major bottleneck.

By Soumia Bouyahiaoui, Manel Kara laouar, Aicha Boutorh, Mohamed Hadj Ameur

llms efficiency safety

arXiv AI

Jun 30

Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

arXiv:2606. 29563v1 Announce Type: cross Abstract: Large language models (LLMs) excel at complex tasks like question answering and summarization, thanks to their ability to handle long-context inputs.

By Shuvendu Roy, Mengyao Zhai, Hossein Hajimirsadeghi, Golnoosh Samei

llms nlp efficiency

arXiv Machine Learning

Aug 26

PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression

arXiv:2608.23843v1 Announce Type: new Abstract: Long-context inference in large language models (LLMs) is increasingly limited by the memory required for the key-value (KV) cache. KV cache compressio...

By Zizhong Wang, Jieying Wang, Zhao Zhang, Jiajia Li

llms efficiency benchmarks

One email a morning, machine-written

Related stories

PatchKV: Weight-Space Compensation of KV Cache

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

SelKV: Selective KV Cache Merging with Per-Token Merge-or-Drop and Attention Compensation

Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression