← Back to all news
arXiv Machine Learning By Linze Wu, Xinrui Chen

Parser States Already Know: Structure-Conditioned KV Persistence for Structured Generation

Read the original on arXiv Machine Learning →

The paper introduces PASK (Parser-Aware Structural KV Persistence), a method that leverages parser transitions to inform key‑value (KV) persistence decisions in structured generation tasks. By aligning KV compression with task‑level structured risk, PASK sets protection floors based on error sensitivity and allocates remaining KV capacity using attention‑output distortion, producing a lightweight, structure‑conditioned lookup policy. In experiments on Qwen3‑4B, PASK achieves a 17.39‑point accuracy gain over the best compressed baseline, while delivering up to 2.2× higher throughput, 3.3× lower TPOT, and 0.53× the peak GPU memory of full KV.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

Related stories

arXiv AI
Oct 28

DCKV: Learned Transform Coding of the Prompt KV Cache in Hybrid Language Models

The paper introduces DCKV, a differentiable codec designed to compress the key-value (KV) cache used in hybrid language models. DCKV jointly optimizes its transform, quantizer, and entropy models—conditioned on prompt tokens and neighboring positions—directly against a frozen language model, balancing compression rate with minimal impact on output quality. Experiments on Qwen3.5-2B demonstrate that DCKV can reduce the storage requirement for a compressed token by over 55× compared to bf16, with a loss of no more than 1.8 points on long‑context benchmarks, and by over 100× with a loss of no more than 3.6 points.

By Egor Maximov, Aleksei Merkelov, Vasiliy Alekseev, Ivan Krivorotov, Valentin Beliaev, Nikita Arsenin, Dmitry Bocharov, Aleksei Goncharov
llmsefficiencybenchmarks
More like this →
Hugging Face Trending Papers
Oct 5

DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression

DeferKV rethinks when to evict key‑value (KV) cache entries in long‑context large language models. By delaying eviction until the first decoding step and combining prompt‑side and decode‑side attention signals, it aligns KV importance estimation with actual generation needs. The method requires no extra training or modules and consistently improves performance on benchmarks while keeping latency low.

llmsefficiency
More like this →