arXiv Computation and Language August 28, 2026 By Hong Chen, Yudong Zeng, Yongwei Huang, Zuhao Ouyang, Junyan Zhang, Xuming Hu

TwinKV: A Composable Repair Pass for KV Cache Eviction via Pairwise Key Redundancy

Read the original on arXiv Computation and Language →

TwinKV is a training‑free, attention‑free repair pass that identifies and swaps orphaned and redundant tokens in a KV cache, improving long‑context inference for small models. It works by detecting near‑duplicate keys and can be composed with existing eviction policies without altering their scoring rules. Experiments on Qwen3‑4B and Llama‑3.2‑1B across LongBench, LooGLE, RULER, and MMLU‑Pro show that TwinKV consistently improves performance for most configurations, especially at tighter compression ratios.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

arXiv Machine Learning

Jul 15

Forgetful Attention: A Trainable Support-Vector Memory with Certified Selection and Exact Unlearning

arXiv:2607. 12204v1 Announce Type: new Abstract: Attention can be viewed as an online learner over context, yet existing test-time memories cannot certify that dropping a token leaves outputs unchanged or delete its influence outright.

By Vishwajith Ramesh

llms rag

arXiv AI

Aug 13

LinearKV: One Cached State Suffices for Position-Independent Caching in Hybrid LLMs

arXiv:2608. 11231v1 Announce Type: new Abstract: LLM serving is increasingly accelerated by position-independent caching (PIC).

By Yirui Liu, Ruoling Qi, Longwen Wang, Xuaner Wu, Jian Chen, Yuxin Jin, Jiawei Shao, Xuelong Li

llms efficiency

arXiv Computation and Language

Aug 28

NestedKV: Nested Memory Routing for Long-Context KV Cache Compression

NestedKV is a training‑free key‑only KV cache compression technique for long‑context language models that uses global, block‑level, and sliding‑window key anchors to score tokens via multi‑time‑scale cosine anomaly. It combines these rankings with a head‑adaptive outer learner and surprise‑gated token routing, requiring no model modification or additional training. Experiments on Qwen3 and Llama‑3.2 across benchmarks such as RULER, LongBench, and MMLU‑Pro show that NestedKV outperforms existing methods when the retained cache is small, achieving up to 19‑point gains on RULER and LongBench at a 75% retention rate.

By Hong Chen, Xiang Liu, Yubo Gao, Yuxuan Fan, Bo Wang, Yuanlin Chu, Yuanguo Lin, Xuming Hu

llms efficiency benchmarks

arXiv AI

Jul 7

IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference

arXiv:2605. 25475v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly expected to operate over long contexts, yet standard softmax attention incurs a KV cache that grows linearly with sequence length, quickly becoming the bottleneck for long context inference.

By Xintong Yang, Hao Gu, Binxing Xu, Lujun Li, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Yike Guo, Sirui Han

llms efficiency

arXiv Computation and Language

3d ago

GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs

The paper introduces GRKV, a training‑free method for compressing the key‑value cache in long‑context large language models. GRKV uses ridge‑regression to redistribute information from evicted tokens to retained ones, aiming to minimize the difference between compressed‑cache and full‑cache attention outputs. Experiments on LongBench and RULER show that GRKV improves overall performance with minimal overhead compared to other merging methods.

By Junjie Peng, You Wu, Haoyi Wu, Jialong Han, Xiaohua Xie, Kewei Tu, Jianhuang Lai

llms efficiency benchmarks

arXiv Machine Learning

Aug 4

RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction

arXiv:2608. 01247v1 Announce Type: cross Abstract: Query-agnostic KV cache eviction compresses a context once and reuses the resulting cache for arbitrary future queries, but performance can collapse under tight budgets.

By Changwoo Baek, Seungjun Shin, Kyeongbo Kong

fine-tuning efficiency benchmarks