arXiv:2609.12475v1 Announce Type: new
Abstract: Comprehensive benchmark suites are essential for improving large language models (LLMs), but many widely used benchmarks are redundant, making evaluati...
By Zhongzhan Huang, Junxin Li, Guoming Ling, Yupei Lin, Shanshan Zhong, Hefeng Wu
arXiv:2505.10202v2 Announce Type: replace
Abstract: Large Language Models (LLMs) have achieved remarkable success but face significant computational and memory challenges, particularly due to their e...
By Jintian Shao, Hongyi Huang, Jiayi Wu, YiMing Cheng, ZhiYu Wu, You Shan, MingKai Zheng
The paper proposes replacing dense output projection in large language models with an HNSW-based vector index to perform maximum inner product search over token embeddings. This approach reduces memory bandwidth usage by retrieving only a small set of high-scoring tokens and can be integrated into existing decoding pipelines via sparse logits scattering. Experiments on Gemma 3, Llama 3.2, and Qwen 3 show up to 82% speed‑up in batch‑size‑one decoding while maintaining generation quality.
By Martin Loretz, Sepp Hochreiter
Celty introduces a co-designed sparse format, GPU kernel, and SIMT microarchitecture to efficiently handle Sparse Matrix‑Sparse Vector (SpMSpV) workloads in large language model inference. Its Run‑Length Compressed CSC (RLC‑CSC) format allows vectorized loading of compressed weight columns and skips memory accesses by exploiting both weight pruning and activation sparsity. The Celty Sparse SIMT Core adds a pipelined RLC decoder that eliminates software index reconstruction and uses local registers for conflict‑free accumulation, achieving up to 5.3× speedup over cuBLAS at 70% dual‑sparsity.
By Ruokai Yin, Priyadarshini Panda
Ripple-Pivot Search (RPS) is a training‑free decoding method for Diffusion Large Language Models that identifies mid‑entropy pivot positions to reduce uncertainty across remaining masked tokens. By proactively committing these pivots and evaluating token assignments via lookahead, RPS enables more tokens to be unmasked in parallel, speeding up decoding. Experiments on three dLLMs and four reasoning/code‑generation benchmarks show 4–10× wall‑clock speedup over standard decoding, up to 18× with KV caching, while maintaining or improving generation quality.
By Yushi Ye, Xu Chen, Haoyun Jiang, Jinsong Lan, Haihong Tang, Xiangtao Li, Mingming Gong, Ivor Tsang, Yanfeng Wang, Jiangchao Yao
arXiv:2608. 13076v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in natural language understanding and generation, but their deployment is constrained by high computational demands.
By Divya Jyoti Bajpai, Kishan Kumar Upadhyay, Manjesh Kumar Hanawal