Hugging Face Trending Papers August 4, 2026

LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs

Read the original on Hugging Face Trending Papers →

Large Language Models (LLMs) are integrated into software systems and AI services, making efficient LLM serving a concern for software engineering. Serving LLMs is challenging because inference requires computation, memory, GPU resources, and execution while maintaining latency and throughput.

Summary generated by The Flow from the publisher's feed. The full article lives at Hugging Face Trending Papers.

arXiv AI

Aug 5

LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs

arXiv:2608. 03036v1 Announce Type: cross Abstract: Large Language Models (LLMs) are integrated into software systems and AI services, making efficient LLM serving a concern for software engineering.

By Forough Majidi, Mohammad Mehdi Morovati, Foutse Khomh, Heng Li

llms reinforcement-learning efficiency multimodal

arXiv AI

1d ago

From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems

arXiv:2608. 15127v1 Announce Type: cross Abstract: Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and persistent state.

By Chaokun Chang, Yukun Zhou, Kaihua Fu, Dakai An, Tianyu Feng, Hanfeng Lu, Sheng Yao, Pu Guo, Yinghao Yu, Yizhou Shan, Bo Li, Binhang Yuan, Wei Wang

llms agents benchmarks

arXiv Machine Learning

Jun 9

OpenCompass: A Universal Evaluation Platform for Large Language Models

arXiv:2605. 19276v3 Announce Type: replace-cross Abstract: In recent years, the field of artificial intelligence has undergone a paradigm shift from task-specific small-scale models to general-purpose large language models (LLMs).

By Maosong Cao, Kai Chen, Haodong Duan, Yixiao Fang, Zhiwei Fei, Tong Gao, Ge Jiaye, Mo Li, Hongwei Liu, Junnan Liu, Yuan Liu, Chengqi Lyu, Han Lyu, Ningsheng Ma, Zerun Ma, Yu Sun, Zhiyong Wu, Linchen Xiao, Zhuozhi Xiong, Jun Xu, Haochen Ye, Zhaohui Yu, Yike Yuan, Songyang Zhang, Yufeng Zhao, Fengzhe Zhou, Peiheng Zhou, Dongsheng Zhu, Lin Zhu, Jingming Zhuo

llms benchmarks

arXiv AI

Jun 30

Scalable Synthesis of distributed LLM workloads through Symbolic Tensor Graphs

arXiv:2511. 10480v3 Announce Type: replace-cross Abstract: Optimizing the performance of large language models (LLMs) on large-scale AI training and inference systems requires a scalable and expressive mechanism to model distributed workload execution.

By Changhai Man, Joongun Park, Hanjiang Wu, Huan Xu, Srinivas Sridharan, Tushar Krishna

llms

arXiv Machine Learning

Jul 14

ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory

arXiv:2607. 11126v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly rely on external tools served by shared providers and accessed by heterogeneous downstream agents.

By Yue Fang, Zhibang Yang, Fangkai Yang, Xiaoting Qin, Liqun Li, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

llms agents benchmarks

arXiv AI

5d ago

SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference

arXiv:2608. 13076v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable success in natural language understanding and generation, but their deployment is constrained by high computational demands.

By Divya Jyoti Bajpai, Kishan Kumar Upadhyay, Manjesh Kumar Hanawal

llms nlp

Related stories

LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs

From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems

OpenCompass: A Universal Evaluation Platform for Large Language Models

Scalable Synthesis of distributed LLM workloads through Symbolic Tensor Graphs

ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory

SPADE: Speculative Decoding for Precise and Low Cost Distributed Edge Cloud Inference