arXiv Machine Learning June 9, 2026 By Anay Chauhan, Gurucharan Marthi Krishna Kumar, Arion Das, Amit Dhanda, Vinija Jain, Aman Chadha, Amitava Das

SPHERICAL KV: Angle-Domain Attention and Rate-Distortion Retention for Efficient Long-Context Inference

Read the original on arXiv Machine Learning →

arXiv:2605. 18856v3 Announce Type: replace Abstract: Long-context inference is increasingly constrained by the KV cache: resident memory grows with context length, and decoding becomes limited by repeated High Bandwidth Memory (HBM) streaming rather than arithmetic.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

efficiency

arXiv AI

3d ago

ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference

arXiv:2602.08005v2 Announce Type: replace-cross Abstract: Efficient long-context inference faces two coupled bottlenecks: KV-cache memory grows linearly with context length, while attention computati...

By Jitai Hao, Qiang Huang, Yaowei Wang, Min Zhang, Jun Yu

llms efficiency multimodal

arXiv AI

Aug 5

Output-Aware Rotation for INT2 KV-Cache Quantization

arXiv:2608. 02691v1 Announce Type: cross Abstract: The key-value (KV) cache has become a major memory and bandwidth bottleneck in long-context large language model inference, making ultra-low-bit quantization increasingly important.

By Vincent-Daniel Yun, Woosang Lim, Minsoo Cheong, Sunwoo Lee, Murali Annavaram, Sai Praneeth Karimireddy, Sungjoo Yoo

llms efficiency benchmarks

arXiv Machine Learning

Aug 26

PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression

arXiv:2608.23843v1 Announce Type: new Abstract: Long-context inference in large language models (LLMs) is increasingly limited by the memory required for the key-value (KV) cache. KV cache compressio...

By Zizhong Wang, Jieying Wang, Zhao Zhang, Jiajia Li

llms efficiency benchmarks

arXiv AI

Jun 9

End-to-End Context Compression at Scale

arXiv:2606. 09659v1 Announce Type: cross Abstract: Long-context language model inference is bottlenecked by memory, as the KV cache grows with context length.

By Ang Li, Sean McLeish, Haozhe Chen, Nimit Kalra, Zaiqian Chen, Artem Gazizov, Venkata Anoop Suhas Kumar Morisetty, Bhavya Kailkhura, Harshitha Menon, Zhuang Liu, Brian R. Bartoldson, Tom Goldstein, Sanae Lotfi, Micah Goldblum, Pavel Izmailov

llms rag agents efficiency

arXiv AI

Jul 8

DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression

arXiv:2607. 06523v1 Announce Type: new Abstract: Long-context language model inference is increasingly limited by the memory bandwidth and capacity required to store key-value caches, yet existing compression methods often apply uniform budgets across layers or tokens and degrade retrieval when lexical cues and semantic states require different preservation.

By Anna Cordoba, Adam Puente Tercero, Nerea Angulo Hijo, Mar Linares Tercero, Julia Barrientos, Ainhoa Miranda, Jesus Olivera

llms nlp efficiency benchmarks

arXiv AI

Jun 9

From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs

arXiv:2606. 09508v1 Announce Type: new Abstract: Existing sparse attention and KV cache compression methods for long-context LLM inference typically apply fixed sparsity patterns or uniform budgets across all attention heads, overlooking the substantial variation in attention behavior among heads and contexts.

By Zhanchao Xu, Haoyang Li, Qingfa Xiao, Fei Teng, Chen Jason Zhang, Lei Chen, Qing Li

llms efficiency

One email a morning, machine-written

Related stories

ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference

Output-Aware Rotation for INT2 KV-Cache Quantization

PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression

End-to-End Context Compression at Scale

DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression

From Rigid to Dynamic: Entropy-Guided Adaptive Inference for Long-Context LLMs