arXiv:2606. 01502v1 Announce Type: cross Abstract: Frontier LLMs increasingly decide what a query attends to with a sparse-attention indexer that picks a few KV-cache blocks per query: attention's unit is now a small, reusable chunk.
By Bole Ma, Jan Eitzinger, Harald K\"ostler, Gerhard Wellein
Page-EntroKV is a new KV‑cache eviction framework designed for grouped‑query attention (GQA) in long‑context autoregressive language models. It pools token importance scores across query heads within each physical group using sink‑isolated Renyi‑2 entropy, then projects these pooled scores onto page frames and performs eviction at the hardware tuple level. The authors prove theoretical guarantees on union overhead, budget preservation, and needle‑retention, and demonstrate on a Qwen2.5‑1.5B‑Instruct pilot that Page‑EntroKV achieves exact union overhead control, eliminates sink masquerade, and retains 100% of critical tokens compared to mean‑pooling baselines.
By Inbasekaran S
arXiv:2609.20276v2 Announce Type: replace-cross
Abstract: Memoizing an expensive function of a sorted score vector is a data-structure problem before it is a numerical one: at a billion gridpoints, a...
By Tamal Maharaj
HAServe introduces a horizon-aware scheduling and memory co-design framework for efficient large language model (LLM) serving. By modeling request admission and key-value (KV) retention as a unified problem using the HorizonSchema abstraction, it selects joint admission and KV-retention candidates that minimize predicted latency across varying prefix structures and arrival rates. Experiments on Qwen3-4B-Instruct-2507 and Llama2-13B with eight H100 GPUs demonstrate up to 5.90× P99 TTFT speedups, 1.44× throughput gains, and a 20.59 percentage point increase in KV-cache hit rate over the strongest baseline.
By Xingyu Qu, Tianhao Lin, Yiqi Li, Zhiyu Chen, Sheng Wang
The paper introduces PACE, a per‑step admission controller that manages intra‑request parallelism in large language model serving. By admitting extra decoding branches only when the predicted branch externality fits within the batch’s current slack, PACE avoids the pitfalls of eager or fixed‑cap branch admission. Experiments on Qwen3‑32B and other large models show that PACE boosts goodput by up to 2.15× while maintaining 99% SLO attainment.
By Swapnil Gandhi, Siva Kumar Sastry Hari, William J. Dally, Christos Kozyrakis
arXiv:2607. 15516v1 Announce Type: cross Abstract: Production LLM deployments combine two cost-reduction primitives: prompt caching (a discounted rate for re-used token prefixes) and prompt compression (fewer tokens sent).
By Yan Song
arXiv:2608. 16309v1 Announce Type: cross Abstract: Static pruning is widely used to accelerate sparse neural retrieval, yet existing studies each validate their conclusions within a single custom pipeline, leaving it unclear which findings transfer to modern engines with different index organizations and dynamic pruning mechanisms.
By Zirui Song, Yuye Zhu, Yang Yang
arXiv:2606. 13126v1 Announce Type: cross Abstract: Retrieval-augmented and agentic workloads repeatedly prefill recurring predictable structured inputs (which we call "spans") such as documents and code files.
By Nathan Ordonez (IBM Research), Thomas Parnell (IBM Research)
The paper investigates when the ordering of deletion requests in machine‑learning model updates can be simplified. It introduces two reductions—position additivity and suffix localization—that transform the permutation objective into more tractable forms such as assignment or sorting. The authors analyze shared affine updates, identify quadratic interactions that hinder additivity, and demonstrate that suffix‑conditioned assignment improves approximation rates, with experiments confirming the presence of these structures in real update sequences.
By Xinyu Wang, Ziyu Zhao, Yixuan He, Xiaowen Chang Alex Smola
arXiv:2607.17469v2 Announce Type: replace-cross
Abstract: How much does an algorithm's running-time distribution under independent randomness reveal about its behavior when independence is no longer...
By Yunbei Xu
arXiv:2610.26938v1 Announce Type: cross
Abstract: Retrieval-Augmented Generation (RAG) is increasingly served across geo-distributed cloud and edge infrastructures, where wide-area latency, cross-reg...
By Hardik Ruparel, Raj Kumar Pandey, Pramod Kumar
The paper presents a table‑free index for tapered memoization grids, enabling compact out‑of‑core evaluation of functions that depend on sorted arguments. By showing that the grid’s key set corresponds to multiset combinations, the authors derive a closed‑form O(d) ranking and unranking scheme that removes the need for large preprocessing tables and allows order‑free parallel construction. The resulting values‑only flat array uses significantly less memory than hash‑map memoization, offers faster query times once cache limits are exceeded, and remains operable with memory‑mapped storage beyond RAM.
By Tamal Maharaj