arXiv Machine Learning August 5, 2026 By Taekyung Heo, Rasoul Shafipour, Ritchie Zhao, Maximilian Golub, Mohammad Mahdi Kamani, Ritika Borkar, Makesh Tarun Chandran, Pantea Zardoshti, Bita Darvish Rouhani

Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

Read the original on arXiv Machine Learning →

arXiv:2608. 03893v1 Announce Type: new Abstract: Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv Machine Learning.

arXiv AI

Aug 11

RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation

arXiv:2608. 08684v1 Announce Type: cross Abstract: Long-context LLM inference is bottlenecked by KV cache memory, yet distributing a limited cache budget across layers remains challenging.

By Dongjie Xu, Kai Qian, Julius, Weijie Shi, Yuxuan Sun, Minghua Tang, Fenglei Jin, Hanchi Dong, Jiajie Xu

llms efficiency

arXiv AI

Jun 16

PolyKV: Heterogeneous Retention and Allocation for KV Cache Compression

arXiv:2606. 15157v1 Announce Type: cross Abstract: KV cache compression is essential for reducing the memory cost of long-context large language model inference.

By Chao Fei, Panos Kalnis

llms efficiency

arXiv AI

Jul 22

CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation

arXiv:2602. 08686v3 Announce Type: replace-cross Abstract: Prefill-only KV compression freezes a token subset at the end of prefill and decodes from it without further eviction.

By Ning Yang, Chengzhi Wang, Yibo Liu, Baoliang Tian, Haijun Zhang

benchmarks

arXiv Machine Learning

Jul 20

Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers

arXiv:2607. 15456v1 Announce Type: new Abstract: Looped, weight-tied Transformers reduce parameters by reusing a block, but decoding still stores a separate K/V cache for every recurrence step.

By James O' Neill, Fergal Reid

llms efficiency

arXiv Machine Learning

Jul 31

Back from the Future: Key-Value Cache Management by Counter-Causal Surprise

arXiv:2607. 27600v1 Announce Type: new Abstract: Key-value (KV) cache management through compression and eviction strategies has emerged as an important research direction in recent years.

By Stephen Gould, Anton van den Hengel

llms efficiency multimodal benchmarks

arXiv Machine Learning

Jun 15

Sub-Token Routing for KV Cache Compression

arXiv:2604. 21335v3 Announce Type: replace Abstract: Transformer inference often requires a large KV cache, especially for long-context language modeling and multimodal generation.

By Wei Jiang, Wei Wang

llms efficiency multimodal

Related stories

RippleKV: Cross-Layer KV Cache Allocation via Perturbation Propagation

PolyKV: Heterogeneous Retention and Allocation for KV Cache Compression

CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation

Looped Latent Attention: Cross-Loop KV Compression for Looped Transformers

Back from the Future: Key-Value Cache Management by Counter-Causal Surprise

Sub-Token Routing for KV Cache Compression