EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
Read the original on arXiv Computation and Language →EntropyCache is a training‑free KV caching technique for diffusion‑based large language models that uses the maximum entropy of newly decoded token distributions to decide when to recompute cached states. By leveraging two empirical observations—entropy correlating with cache drift and feature volatility persisting across multiple steps—EntropyCache makes a skip‑or‑recompute decision with only O(V) computation per step, independent of context length or model size. Experiments on LLaDA‑8B‑Instruct and Dream‑7B‑Instruct demonstrate speedups of 15.2×–26.4× on standard benchmarks and 22.4×–24.1× on chain‑of‑thought benchmarks while maintaining competitive accuracy and keeping decision overhead below 0.5% of inference time.
Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.