arXiv:2609.05760v1 Announce Type: cross
Abstract: We present RAGMark, a modular benchmarking framework for advanced Retrieval-Augmented Generation (RAG) systems targeting small-scale multi-GPU enviro...
By Zlatan Feric, Amir Taherin, Bin Ren, Yanzhi Wang, Jennifer Dy, David Kaeli
arXiv:2603. 29002v3 Announce Type: replace-cross Abstract: Modern large language models (LLMs) increasingly depends on efficient long-context processing and generation mechanisms, including sparse attention, retrieval-augmented generation (RAG), and compressed contextual memory, to support complex reasoning.
By Zifan He, Rui Ma, Yizhou Sun, Jason Cong
The paper introduces QUILT, a new sparse‑attention execution mechanism that jointly processes neighboring queries to reuse shared key‑value (KV) entries, thereby reducing redundant memory traffic and computation. QUILT employs Shift‑and‑Compare Set Decomposition (SCSD) to convert irregular set operations into regular data‑parallel primitives and pipelines this with attention computation to hide overhead. The approach includes cascaded sharing across multiple granularities and a tile‑aware strategy that balances sharing with hardware utilization, achieving up to 55.1% lower kernel latency and 36.8% faster time‑to‑first‑token on LongBench benchmarks with negligible accuracy loss.
By Zhenduo Zhao, Qihui Zhou, Mingcong Song, Zhiyi Chen, Chuangguan Ye, Fengfan Hou, Zequn Gong, Jing Li, Hongjie Si, Guoping Long
arXiv:2609.21281v1 Announce Type: cross
Abstract: Embedding-based retrieval on user-generated content at the trillion-document scale exposes a sharp conflict between two production demands: deep, exp...
By Hao Fu, Jichao Sun, Baiting Zhu, Qiaoling Liu, Yan Shi, Cheng Lu, Liu Liu, Yubo Wang, Xin Yao, Xiangyu Niu, Xu Dong, Wenhan Lyu, Chiyao Shen, Yinjie Huang, Minglei Chen, Shuai Ding, Li Fan, Xiao Kong
arXiv:2602. 22647v2 Announce Type: replace-cross Abstract: Generative retrieval has emerged as a powerful paradigm for LLM-based recommendation.
By Zhengyang Su, Isay Katsman, Yueqi Wang, Ruining He, Lukasz Heldt, Raghunandan Keshavan, Shao-Chuan Wang, Xinyang Yi, Mingyan Gao, Onkar Dalal, Lichan Hong, Ed Chi, Ningren Han
STAPLE is a method that integrates static sparse memory into pretrained language models by adding token‑indexed embeddings as a residual pathway while preserving the original feed‑forward network. On the OLMo‑2 1B and 7B models, STAPLE outperforms dense training and other static sparse baselines, improving benchmark scores by up to 4.27 points and achieving faster decoding with less GPU memory and energy usage. The approach demonstrates that large pretrained models can be expanded efficiently without costly sparse pretraining from scratch.
By Ranajoy Sadhukhan, Euijin Hong, Sheng Cao, Zechun Liu, Shuowei Jin, Sha Li, Changlong Yu, Xin Liu, Qin Lu, Zhenyu Shi, Priyanka Nigam, Beidi Chen