Hugging Face Trending Papers October 5, 2026

DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression

Read the original on Hugging Face Trending Papers →

DeferKV rethinks when to evict key‑value (KV) cache entries in long‑context large language models. By delaying eviction until the first decoding step and combining prompt‑side and decode‑side attention signals, it aligns KV importance estimation with actual generation needs. The method requires no extra training or modules and consistently improves performance on benchmarks while keeping latency low.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at Hugging Face Trending Papers.

arXiv AI

Aug 25

KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference

arXiv:2608.21362v1 Announce Type: new Abstract: Transformer-based large language models (LLMs) incur high prefill latency because key-value (KV) tensors must be recomputed for each request. Existing...

By Srihari Unnikrishnan

llms efficiency

arXiv AI

Jul 28

MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models

arXiv:2607. 22586v1 Announce Type: new Abstract: Key-Value (KV) caching is essential for efficient inference in multimodal large language models (MLLMs), yet its memory footprint grows linearly with context length and becomes a major bottleneck due to the large number of visual tokens.

By Jinsong Shu, Chenyang Wu, Zhongle Xie, Baokun Wang, Lidan Shou

llms efficiency multimodal benchmarks

arXiv Machine Learning

Jun 29

Learning to Evict from Key-Value Cache

arXiv:2602. 10238v2 Announce Type: replace-cross Abstract: The growing size of Large Language Models (LLMs) makes efficient inference challenging, primarily due to the memory demands of the autoregressive Key-Value (KV) cache.

By Luca Moschella, Laura Manduchi, Ozan Sener

llms agents reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Aug 26

PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression

arXiv:2608.23843v1 Announce Type: new Abstract: Long-context inference in large language models (LLMs) is increasingly limited by the memory required for the key-value (KV) cache. KV cache compressio...

By Zizhong Wang, Jieying Wang, Zhao Zhang, Jiajia Li

llms efficiency benchmarks

arXiv Machine Learning

Sep 10

MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference

arXiv:2609.07966v1 Announce Type: new Abstract: Key--value (KV) cache compression is an effective way to reduce the memory overhead of large language model (LLM) inference, particularly for long-cont...

By Michael Wang, Keith Li, Roozbeh Bostandoost

llms efficiency

arXiv AI

Jun 30

Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

arXiv:2606. 29563v1 Announce Type: cross Abstract: Large language models (LLMs) excel at complex tasks like question answering and summarization, thanks to their ability to handle long-context inputs.

By Shuvendu Roy, Mengyao Zhai, Hossein Hajimirsadeghi, Golnoosh Samei

llms nlp efficiency

One email a morning, machine-written

Related stories

KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference

MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models

Learning to Evict from Key-Value Cache

PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression

MetaKV: Adaptive KV Cache Compression for Constrained LLM Inference

Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM