CORAL-KV is a new framework for compressing the key‑value (KV) cache used by autoregressive video diffusion models. It predicts keys and values across cache layers using block‑diagonal‑plus‑low‑rank (block‑DPLR) predictors, then quantizes only the residuals, offering a flexible trade‑off between fidelity and memory usage. In extensive tests on HY‑WorldPlay and MAGI‑1, the reusable configuration outperforms QVG‑4 by up to 4.95 dB PSNR while reducing cache memory by 37–48 %, and the video‑adapted configuration surpasses QVG‑Pro by 3.77 dB PSNR with 33.7 % less memory, all while maintaining comparable latency and reducing temporal discontinuities.
By Albert Ge, Akash Haridas, Utkarsh Saxena, Ruijing Yang, Ian Colbert, Frederic Sala, Vikram Appia, Emad Barsoum
DensityKV is a training‑free strategy for managing the historical key‑value (KV) cache in autoregressive video diffusion models. It creates a separate token‑level KV bank for each attention head and uses Soft‑Riesz density to measure and limit local redundancy among post‑RoPE keys, thereby preventing the KV archive from growing indefinitely. Experiments on three video generation backbones demonstrate that, with the same KV capacity limit, DensityKV improves long‑horizon consistency and generation stability while keeping persistent storage bounded regardless of rollout length.
By Wenqu Zhao, Xuemin Chi, Xin Zhang, Guoqing Ma, Baorun Li, Jianjie Fang, Peizhi Tang, Chen Gao, Wei Wu
arXiv:2610.02953v1 Announce Type: new
Abstract: Long-context LLM serving is increasingly bottlenecked by KV-cache memory, especially in resource-constrained scenarios. Among existing KV-cache compres...
By Zihan Teng, Jiayu Zhao, Wentao Ren, Minhao Fan, Tianrui Ma, Song Chen, Weichen Liu
The paper introduces GAAC, a lossless key‑value (KV) compression method for large language model inference on personal devices. GAAC encodes BF16 exponents with per‑channel base‑plus‑offset codes, adapts code widths across small groups, and reconstructs KV tiles on the fly to avoid full‑cache decompression. Experiments show 1.4× compression for BF16 KV, speedups of up to 1.27× in decode attention, and a 26% reduction in resident KV memory while boosting throughput on NVIDIA and Apple hardware.
By Yipin Guo, Arun M George, Siddharth Joshi
arXiv:2610.06927v1 Announce Type: cross
Abstract: The key-value (KV) cache of autoregressive transformers grows linearly with context length and dominates memory at long context. Most training-free r...
By Sara Abdali, Jongwoo Ko, Pashmina Cameron
arXiv:2609.26425v1 Announce Type: new
Abstract: KV cache memory has become a major deployment bottleneck for video generation and world models, which motivates low-bit quantization study for efficien...
By Jiaqi Zhao, Xiaobin Hu, Bo Yin, Junpeng Jiang, Miao Zhang, Shuicheng Yan
arXiv:2606. 01790v1 Announce Type: cross Abstract: Vision-language-model-based graphical user interface (GUI) agents have shown broad automation capabilities, yet deployment is bottlenecked by a key-value (KV) cache that grows linearly with interaction steps.
By Yuhang Han, Wenzheng Yang, Yujie Chen, Xiangqi Jin, Yaojie Zhang, Siteng Huang, Linfeng Zhang
arXiv:2610.32784v1 Announce Type: new
Abstract: The rapid growth of video data has created an urgent need for efficient video understanding systems that can handle long and continuous visual inputs,...
By Yonghui Niu, Yuhao Ji, Chang Nie, Jiao Zhang, Yilong Li, Junlan Feng, Ran He, Caifeng Shan, Chaoyou Fu
arXiv:2607. 01520v1 Announce Type: new Abstract: Transformer inference on long sequences is expensive because softmax attention repeatedly reads from a large KV cache.
By Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno, Sebastian Trimpe, Marco Pavone
arXiv:2611.05380v1 Announce Type: new
Abstract: Visual autoregressive generation spans two dominant paradigms: next-scale prediction for images and video, and next-frame prediction for video. Both de...
By Boxun Xu, Jiaji Lu, Yuxuan Yin, Zihu Wang, Ziyue Liu, Yu Wang, Zirui Liu, Peng Li
arXiv:2609.39096v1 Announce Type: new
Abstract: Autoregressive video diffusion supports streaming generation and interactive control, but its KV cache grows with the generated history. Existing compr...
By Zeqi Xiao, Qingle Liu, Kaiwen Zhang, Yifan Zhou, Zihan Ding, Xingang Pan
Minima-KV introduces a retention‑preserving hierarchy for mixed‑format paged attention that keeps recent and protected anchor pages in FP8 while older pages are compressed into packed TQ3, allowing every live‑request page to remain addressable. The approach uses format‑specific kernels and a globally normalized online‑softmax merge to compute partial attention states, enabling direct heterogeneous decoding without a dense shadow cache. Experiments on Qwen3.6‑27B on a 96‑GB NVIDIA RTX PRO 6000 Blackwell GPU show 3.50× compression over BF16 and 1.75× over FP8, with minimal impact on performance across long‑context benchmarks.
By Sergii Kozyrev (Minima AI, Inc), Davyd Maiboroda (Minima AI, Inc)