iS-KV: Online Low-Rank KV Cache Compression via Block-Incremental SVD
Read the original on arXiv AI →The paper introduces iS-KV, an online low‑rank KV‑cache compression technique that uses block‑incremental SVD to manage memory during long‑horizon autoregressive decoding. Unlike token‑eviction methods, iS‑KV retains all positions in a compact representation by keeping a recent window exact and incrementally folding older states into bounded‑rank bases, synchronizing coordinates as the basis evolves. Experiments on DeepSeek‑R1‑Distill‑Llama‑8B and Qwen3‑8B show that iS‑KV achieves high accuracy (82.6% and 89.2% respectively) while providing 4.06‑fold and 5.64‑fold compression, outperforming token‑eviction baselines under matched memory budgets.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.