arXiv Machine Learning September 7, 2026 By Janghyeon Kim, Minsoo Kim, Kyuhong Shim, Jungwook Choi

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

Read the original on arXiv Machine Learning →

BeaconKV is a training‑free key‑value cache compression technique for Large Reasoning Models that uses beacon queries—compact representatives of query clusters—to predict which KV pairs will be revisited during long‑horizon reasoning. By focusing on Thought Revisiting Tokens that re‑attend distant context, BeaconKV reduces memory usage up to 5.8× and improves throughput by over 4.3× while largely preserving cache accuracy across multiple open‑source LRMs and reasoning benchmarks.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv AI

Jul 3

Kara: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression

arXiv:2607. 01237v1 Announce Type: cross Abstract: Reasoning language models often generate long chain-of-thought (CoT), which accumulates a massive KV cache during the decoding phase and incurs high decoding latency and limited throughput.

By Shen Han, Yuyang Wu

llms efficiency

arXiv AI

Jul 7

KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression

arXiv:2607. 01237v2 Announce Type: replace-cross Abstract: Reasoning language models often generate long chain-of-thought (CoT), which accumulates a massive KV cache during the decoding phase and incurs high decoding latency and limited throughput.

By Shen Han, Yuyang Wu, Junpu Yu, Olexandr Isayev

llms efficiency

arXiv AI

Aug 11

Thought-Level Beam Search for Reasoning

arXiv:2608. 08020v1 Announce Type: new Abstract: Test-time compute scaling is a primary driver of performance in large reasoning models (LRMs), but extreme inefficiency bounds current approaches, shifting the critical question from \emph{how much} compute to spend, to \emph{where} to allocate it.

By Lijie Yang, Hongyin Luo, Tri Dao, Ravi Netravali

efficiency benchmarks

arXiv Machine Learning

Jun 2

ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution

arXiv:2602. 03203v2 Announce Type: replace-cross Abstract: Recently, large language models (LLMs) have shown remarkable reasoning abilities by producing long reasoning traces.

By Zican Dong, Peiyu Liu, Junyi Li, Zhipeng Chen, Han Peng, Shuo Wang, Wayne Xin Zhao

llms reinforcement-learning efficiency benchmarks

arXiv Machine Learning

Jun 3

Value-Aware Stochastic KV Cache Eviction for Reasoning Models

arXiv:2606. 03928v1 Announce Type: new Abstract: Reasoning models improve accuracy through extended chains of thought, but their long outputs create a memory and compute bottleneck.

By Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu, Chenghao Yang, Jesse Thomason, Robin Jia

efficiency benchmarks

arXiv AI

Jun 26

Information-Aware KV Cache Compression for Long Reasoning

arXiv:2606. 26875v1 Announce Type: cross Abstract: Reasoning capability has advanced rapidly in large language models (LLMs), leading to an increasing size of key-value (KV) cache in both prefilling and decoding stages.

By Jushi Kai, Zhuiri Xiao, Alexandra Birch, Zhouhan Lin

llms efficiency benchmarks

One email a morning, machine-written

Related stories

Kara: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression

KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression

Thought-Level Beam Search for Reasoning

ForesightKV: Optimizing KV Cache Eviction for Reasoning Models by Learning Long-Term Contribution

Value-Aware Stochastic KV Cache Eviction for Reasoning Models

Information-Aware KV Cache Compression for Long Reasoning