Compression Does Not End at Selection: Post-Selection Positional Geometry for Sparse KV Caches
Read the original on arXiv Computation and Language →The paper investigates how the positional geometry of cached key‑value (KV) states affects inference quality in sparse KV compression. It introduces RoPERelay, a method that reassigns compact, order‑preserving RoPE coordinates and re‑rotates cached key phases to better align with current queries, without requiring new selection or extra cache budget. Experiments on two models and multiple selectors show that RoPERelay improves answer quality in most cases, notably raising Llama‑3.1‑8B‑Instruct accuracy from 23.6% to 35.4% on LongMemEval‑S with only a 1–3% runtime overhead.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.