arXiv Machine Learning October 28, 2026 By Lin Zhang

SAKI-RoPE: Score-Aware Low-Rank Key Indexing with RoPE-Aware Covariance and Random-Matrix Noise Analysis

Read the original on arXiv Machine Learning →

The paper introduces SAKI-RoPE, a training‑free low‑rank key‑indexing method that optimally preserves attention scores by solving a covariance‑weighted rank‑r problem using query and key statistics. It outperforms key‑PCA across multiple large language models and context lengths, achieving higher recall with significantly lower rank. The authors also analyze random‑matrix noise to justify the calibration token budget and demonstrate that clipping the Marchenko‑Pastur bulk improves performance when calibration data is limited.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

llms

arXiv Machine Learning

Aug 7

SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

arXiv:2608. 03228v2 Announce Type: replace Abstract: Existing low rank KV cache methods preserve either model weights or key variance, neither of which directly reflects the attention scores used during inference.

By Lin Zhang

llms efficiency benchmarks

arXiv Machine Learning

Aug 5

SAKI: Score-Aware Low-Rank Key Indexing for Long-Context KV Retrieval

arXiv:2608. 03228v1 Announce Type: new Abstract: Existing low rank KV cache methods preserve either model weights or key variance, neither of which directly reflects the attention scores used during inference.

By Lin Zhang

llms efficiency benchmarks

arXiv Computation and Language

Oct 1

Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models

arXiv:2410.02343v2 Announce Type: replace Abstract: Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer int...

By Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Anastasia Voznyuk, Andrei Andriiainen, Irina Piontkovskaya, Evgeny Burnaev, Serguei Barannikov

llms rag nlp benchmarks safety

arXiv Computation and Language

Sep 28

Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction

arXiv:2609.30738v1 Announce Type: new Abstract: KV cache eviction methods such as SnapKV and PyramidKV rank tokens solely by mean attention over a small observation window. We study a unified score,...

By Tianfang Xie, Wei Zhu

llms efficiency

arXiv Machine Learning

Jul 20

VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs

arXiv:2607. 15498v1 Announce Type: cross Abstract: The key-value (KV) cache is the main memory bottleneck in long-context large language model (LLM) inference.

By Shahrzad Esmat, Dhawal Shah, Ali Jannesari

llms efficiency benchmarks

arXiv Machine Learning

Sep 2

Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation

arXiv:2609.00762v1 Announce Type: new Abstract: Parameter-efficient fine-tuning is usually framed as a question of how many parameters to update. Under a severe trainable-state budget, however, where...

By Wentao Ye, Zhanming Shen, Zhiqing Xiao, Yao Ding, Haobo Wang, Gang Chen

llms fine-tuning

One email a morning, machine-written

Related stories

SAKI: Score-Aware Low-Rank Key Indexing with Random-Matrix Noise Correction for KV Retrieval

SAKI: Score-Aware Low-Rank Key Indexing for Long-Context KV Retrieval

Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models

Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction

VarRate: Training-Free Variable-Rate KV Cache Compression for Long-Context LLMs

Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation