arXiv Machine Learning August 7, 2026 By Lin Zhang

SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

Read the original on arXiv Machine Learning →

arXiv:2608. 03228v2 Announce Type: replace Abstract: Existing low rank KV cache methods preserve either model weights or key variance, neither of which directly reflects the attention scores used during inference.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

arXiv Machine Learning

Aug 5

SAKI: Score-Aware Low-Rank Key Indexing for Long-Context KV Retrieval

arXiv:2608. 03228v1 Announce Type: new Abstract: Existing low rank KV cache methods preserve either model weights or key variance, neither of which directly reflects the attention scores used during inference.

By Lin Zhang

llms efficiency benchmarks

arXiv Computation and Language

Sep 28

Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction

arXiv:2609.30738v1 Announce Type: new Abstract: KV cache eviction methods such as SnapKV and PyramidKV rank tokens solely by mean attention over a small observation window. We study a unified score,...

By Tianfang Xie, Wei Zhu

llms efficiency

arXiv Machine Learning

Jul 28

DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation

arXiv:2607. 24331v1 Announce Type: new Abstract: As the inference phase of Large Language Models (LLMs) requires handling long context windows, the Key-Value (KV) cache initially appears to address this challenge but eventually becomes a significant bottleneck as the context window continues to grow.

By Tan T. Nguyen, Quan V. Dang

llms efficiency safety

arXiv AI

Sep 10

Attention-Weighted Value Projection for KV-Cache Compression

arXiv:2604.11501v2 Announce Type: replace-cross Abstract: Rank reduction discards dimensions; quantization keeps them at lower precision. Comparing the two requires a choice of what compression shoul...

By Samuel Salfati

efficiency

arXiv Machine Learning

Jul 20

VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs

arXiv:2607. 15498v1 Announce Type: cross Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference.

By Shahrzad Esmat, Dhawal Shah, Ali Jannesari

llms efficiency benchmarks

arXiv Machine Learning

Jul 31

Recall Before You Rank: Similarity-Guided Top-$K$ Reuse for Efficient Long-Context Attention

arXiv:2607. 27692v1 Announce Type: cross Abstract: Top-$K$ sparse attention reduces the cost of Softmax and value aggregation by attending to only a small subset of key--value (KV) entries.

By Wenshuai Yao, Wenyong Zhou, Hanyong Shao, Yizhe Chen, Zhiyuan Ning, Yuannuo Feng, Ru Huang, Kechao Tang

efficiency

One email a morning, machine-written

Related stories

SAKI: Score-Aware Low-Rank Key Indexing for Long-Context KV Retrieval

Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction

DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation

Attention-Weighted Value Projection for KV-Cache Compression

VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs

Recall Before You Rank: Similarity-Guided Top-$K$ Reuse for Efficient Long-Context Attention