arXiv:2606. 26120v1 Announce Type: cross Abstract: Diffusion Large Language Models (dLLMs) offer a promising alternative to autoregressive models, excelling in text generation tasks due to their bidirectional attention mechanisms.
By Tianyi Wu, Xiaoxi Sun, Yanhua Jiao, Yulin Li, Yixin Chen, YunHao Cao, YiQi Hu, Zhuotao Tian
The paper "LLM Inference in a Flash!" proposes an integer‑only quantization scheme and a dictionary‑based KV cache compression technique to enable large language model inference on compute‑in‑flash (CIF) devices. By eliminating floating‑point operations and reducing KV cache traffic through sparse dictionary coding, the authors achieve minimal accuracy loss while cutting dynamic KV cache traffic by 15× on Llama‑3.1‑8B and Qwen‑2.5‑7B models.
By Sebastian Zhao, Minseo Kim, Coleman Hooper, Luca Manolache, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, Amir Gholami
FlashOffloading is a host‑DRAM‑centric inference architecture that fully offloads the key‑value (KV) cache for large language model serving, addressing the high‑bandwidth memory (HBM) capacity wall that limits concurrency. It introduces a rolling HBM cache for prefill, a hot cache for decode, a reverse sparse index with zero miss‑lookup overhead, and a compute–load overlapped kernel to reduce KV offloading latency across the pipeline. Experiments on an Ascend910C platform with GLM‑5.2 show up to 1.38× prefill throughput and 1.91× decode throughput gains under a 30 ms time‑per‑output‑token constraint.
By Xuan Wang, Zhuanhao Wu, Tingzhu Fang, Xushi Li, Sicheng Zhang, Zhenyu Yang, Feng Liu, Zhaohui Fan, Qiang Xie, Huayi Zou, Yuncong Liu, Zeyu Song, Ke Zhu, Yanjia Zeng, Jingchun Gao, Yan Shi, Kui Luo, Gezi Li, Liangjun Feng, Cheng Li
arXiv:2511. 21759v2 Announce Type: replace-cross Abstract: Diffusion-based large language models (dLLMs) have recently gained significant attention for their exceptional performance and inherent potential for parallel decoding.
By Linye Wei, Wenjue Chen, Pingzhi Tang, Xiaotian Guo, Le Ye, Runsheng Wang, Meng Li
FlashOffloading is a host-DRAM-centric inference architecture that fully offloads the key-value cache for large language model serving, addressing the high-bandwidth memory capacity wall that limits concurrency. It introduces a rolling HBM cache for prefill, a hot cache for decode, a reverse sparse index, and a compute–load overlapped kernel to minimize offloading latency across the pipeline. Experiments with GLM‑5.2 on an Ascend910C platform show up to 1.38× higher prefill throughput and 1.91× higher decode throughput under a 30 ms time‑per‑output‑token constraint.
arXiv:2506. 06295v2 Announce Type: replace-cross Abstract: Autoregressive Models (ARMs) have long dominated the landscape of Large Language Models.
By Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, Yichen Zhu, Linfeng Zhang