arXiv Computation and Language
Prefix Sharing Is a Sorting Problem
Read the original on arXiv Computation and Language →The Flow has not summarised this story yet — read it at arXiv Computation and Language.
Related stories
arXiv AI
Jun 2
Move the Query, Not the Cache: Characterizing Cross-Instance Latent Attention Redistribution Across GPU Fabrics
arXiv:2606. 01502v1 Announce Type: cross Abstract: Frontier LLMs increasingly decide what a query attends to with a sparse-attention indexer that picks a few KV-cache blocks per query: attention's unit is now a small, reusable chunk.
arXiv Machine Learning
Sep 23
Memoization Without Keys: Compact, Out-of-Core Tables for Functions of Sorted Arguments
arXiv:2609.20276v2 Announce Type: replace-cross Abstract: Memoizing an expensive function of a sorted score vector is a data-structure problem before it is a numerical one: at a billion gridpoints, a...
arXiv AI
Jul 20
Cache-Aware Prompt Compression:A Two-Tier Cost Model for LLM API Caching
arXiv:2607. 15516v1 Announce Type: cross Abstract: Production LLM deployments combine two cost-reduction primitives: prompt caching (a discounted rate for re-used token prefixes) and prompt compression (fewer tokens sent).
arXiv AI
Aug 18
Static Pruning Across Sparse Retrieval Regimes: What Transfers, What Breaks, and What Still Helps
arXiv:2608. 16309v1 Announce Type: cross Abstract: Static pruning is widely used to accelerate sparse neural retrieval, yet existing studies each validate their conclusions within a single custom pipeline, leaving it unclear which findings transfer to modern engines with different index organizations and dynamic pruning mechanisms.
arXiv AI
Jun 12
MiniPIC: Flexible Position-Independent Caching in <100LOC
arXiv:2606. 13126v1 Announce Type: cross Abstract: Retrieval-augmented and agentic workloads repeatedly prefill recurring predictable structured inputs (which we call "spans") such as documents and code files.
arXiv Machine Learning
1d ago
Local Search with Correlated Randomness
arXiv:2607.17469v2 Announce Type: replace-cross Abstract: How much does an algorithm's running-time distribution under independent randomness reveal about its behavior when independence is no longer...