arXiv AI June 16, 2026 By Yifan Sui, Han Zhao, Rui Ma, Zhiyuan He, Hao Wang, Jianxun Li, Kaiqiang Xu, Kai Chen, Yuqing Yang

Parallelizing Tool Execution and LLM Generation for Low-Latency Agent Serving

Read the original on arXiv AI →

arXiv:2603. 18897v2 Announce Type: replace-cross Abstract: LLM-powered agents execute tasks through a sequential loop of model generation and tool execution.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv Machine Learning

Aug 4

Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale

arXiv:2608. 00101v1 Announce Type: cross Abstract: AI coding agents like GitHub Copilot, Claude Code, and Codex interleave multi-step LLM inference with tool execution, creating a workload different from chatbots.

By Banruo Liu, Haoran Qiu, \'I\~nigo Goiri, Rodrigo Fonseca, Ricardo Bianchini, Esha Choukse

llms agents efficiency

arXiv AI

Jul 28

SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving

arXiv:2607. 23933v1 Announce Type: cross Abstract: As LLM agents increasingly rely on the Model Context Protocol (MCP) to invoke isolated external sandboxes, disaggregated sandbox deployment introduces a fundamental tension between resource utilization and interactive tail latency.

By Yihui Zhang (Beihang University), Tianyu Wo (Beihang University), Jinghao Wang (Beihang University), Xiaoyang Sun (University of Leeds), Menghao Zhang (Beihang University), Cangzhou Yuan (Beihang University), Li Li (Beihang University), Chunming Hu (Beihang University), Albert Y. Zomaya (The University of Sydney), Renyu Yang (Beihang University)

llms rag agents

arXiv AI

1d ago

From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems

arXiv:2608. 15127v1 Announce Type: cross Abstract: Agentic applications are shifting AI serving from isolated model inference to long-running workloads in which LLMs coordinate tools, environments, and persistent state.

By Chaokun Chang, Yukun Zhou, Kaihua Fu, Dakai An, Tianyu Feng, Hanfeng Lu, Sheng Yao, Pu Guo, Yinghao Yu, Yizhou Shan, Bo Li, Binhang Yuan, Wei Wang

llms agents benchmarks

arXiv AI

Jun 30

TraceLab: Characterizing Coding Agent Workloads for LLM Serving

arXiv:2606. 30560v1 Announce Type: cross Abstract: Coding agents are rapidly becoming a major application of agentic LLMs, but serving them efficiently remains challenging.

By Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci

llms agents benchmarks

arXiv Machine Learning

Jul 10

Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems

arXiv:2607. 08010v1 Announce Type: cross Abstract: Production LLM agents often waste latency and reliability by regenerating code for the same procedural steps on every request.

By Kalle Kujanp\"a\"a, Ning Liu, Shahnawaz Alam, Yeshwanth Reddy Sura, Tianyu Yang, Kristina Klinkner, Shervin Malmasi

llms agents benchmarks

arXiv Machine Learning

Jul 14

ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory

arXiv:2607. 11126v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly rely on external tools served by shared providers and accessed by heterogeneous downstream agents.

By Yue Fang, Zhibang Yang, Fangkai Yang, Xiaoting Qin, Liqun Li, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang

llms agents benchmarks

Related stories

Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale

SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving

From LLM Inference to Agentic Workloads: Characterization and Implications for Serving Systems

TraceLab: Characterizing Coding Agent Workloads for LLM Serving

Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems

ToolAtlas: Learning Once, Reusing Everywhere with Tool-Side Memory