← Back to all news
arXiv AI By Saurabh Agarwal, Bodun Hu, Luis Pabon, Myungjin Lee, Jayanth Srinivasa, Aditya Akella

Harmonia: End-to-End RAG Serving Optimization

Read the original on arXiv AI →

arXiv:2505. 07833v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) improves the reliability of large language models by integrating external knowledge, but serving RAG pipelines efficiently is challenging because requests traverse heterogeneous components spanning LLM inference, databases, and CPU-side processing.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Jun 9

OpenCompass: A Universal Evaluation Platform for Large Language Models

arXiv:2605. 19276v3 Announce Type: replace-cross Abstract: In recent years, the field of artificial intelligence has undergone a paradigm shift from task-specific small-scale models to general-purpose large language models (LLMs).

By Maosong Cao, Kai Chen, Haodong Duan, Yixiao Fang, Zhiwei Fei, Tong Gao, Ge Jiaye, Mo Li, Hongwei Liu, Junnan Liu, Yuan Liu, Chengqi Lyu, Han Lyu, Ningsheng Ma, Zerun Ma, Yu Sun, Zhiyong Wu, Linchen Xiao, Zhuozhi Xiong, Jun Xu, Haochen Ye, Zhaohui Yu, Yike Yuan, Songyang Zhang, Yufeng Zhao, Fengzhe Zhou, Peiheng Zhou, Dongsheng Zhu, Lin Zhu, Jingming Zhuo
llmsbenchmarks
More like this →
arXiv AI
Sep 2

ContextPipe: Database-Inspired Context Assembly for Long-Horizon Agents

ContextPipe is a database-inspired framework for assembling context in long-horizon large language model agents. It treats context assembly like relational query execution, using a five-phase pipeline—Plan, Bind, Optimize, Execute, Feedback—backed by a structured catalog, deterministic cache-aware optimizer, and EXPLAIN ANALYZE tracing. In a preliminary evaluation on the SWE-bench Pro Qutebrowser subset, ContextPipe reduced token volume by 31%, LLM calls by 23%, and response time by 9% compared to an append-only policy, though it lowered KV cache-hit ratio.

By Peng Xu, Zuyu Zhang, Yuze Sun, Feng Tian, Long Wang, Chen Zhang
llmsagentsefficiency
More like this →