The paper introduces A-LEMS, a framework that measures AI energy consumption at the goal level rather than per model invocation, defining Energy per Successful Goal (EpG) as total workflow energy divided by successfully completed goals. It also proposes the Orchestration Overhead Index (OOI) to quantify the extra energy cost of agentic systems compared to linear execution, revealing significant overheads across various hardware platforms and task families. The study demonstrates that traditional per-inference metrics underestimate the true energy cost of completing user goals in agentic AI workflows.
By Deepak Panigrahy, Aakash Tyagi, Stephen Abkin, David Kebo Houngninou, Rabi Mahapatra
arXiv:2608. 04458v1 Announce Type: new Abstract: Agentic AI is emerging in datacenters, but its architectural implications remain unexplored.
By Jirong Yang, Peizhe Liu, Chaojie Zhang, Jovan Stojkovic
arXiv:2608. 10714v1 Announce Type: cross Abstract: The Organic 6G vision of a network of networks spanning an edge-cloud continuum complemented by non-terrestrial resources requires, to realize its promise, service provisioning that is simple to operate, scalable across independently administered domains, and agile under domain churn (i.
By Masoud Shokrnezhad, Tarik Taleb
arXiv:2609.17193v1 Announce Type: new
Abstract: Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference, motivating the deployment of LLMs across distributed...
By Zhen Li, Jun Cai, Haoran Gao, An Li, Tan Li
The paper investigates how token length affects compute time and energy usage in Large Language Models, focusing on both prefill (input) and decode (output) phases. It finds that decode energy scales linearly with output length, while prefill energy shows both linear and quadratic scaling with input length, highlighting the importance of large input contexts in agentic workflows. The study also demonstrates that compute time can reliably proxy energy consumption when direct measurements are unavailable, providing a framework for more sustainable AI deployment.
By Lars Krupp, Daniel Gei{\ss}ler, Francisco M. Calatrava-Nicolas, Stefan Gerd Fritsch, Paul Lukowicz, Jakob Karolus
arXiv:2610.33475v1 Announce Type: cross
Abstract: The widespread deployment of foundational Large Language Models (LLM) across cloud datacenters marks a fundamental paradigm shift from stateless deep...
By Jannatun Noor, Athar Noor Mohammad Rafee, Md. Sadman Haque, Sreezon Das Gupta, Atiar Osman, Md. Abu Obaida Zishan
arXiv:2610.07094v1 Announce Type: cross
Abstract: LLM deployment is shifting from single-turn completion to agentic trajectories in which a model plans, calls tools, reads results and reasons at test...
By Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona Zahid
arXiv:2606. 11440v1 Announce Type: new Abstract: Existing multi-agent LLM orchestration methods, ranging from brute-force ensembles to learned routers, select models and topologies based on task and model features.
By Ahasan Kabir, Jiaqi Xue, Mengxin Zheng, Qian Lou
arXiv:2604. 26963v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed as the execution core of autonomous agents rather than as standalone text generators.
By Yifei Wang, Hancheng Ye, Yechen Xu, Cong Guo, Chiyue Wei, Qinsi Wang, Dongting Li, Tingjun Chen, Hai "Helen" Li, Danyang Zhuo, Yiran Chen
The paper introduces Conformal Reservation for Orchestration (CRO), an online algorithm that uses past execution traces to allocate compute and bandwidth for agentic AI systems at dispatch time. CRO provides finite‑sample, distribution‑free coverage guarantees, ensuring that resource reservations meet specified probabilistic deadlines. Simulations demonstrate that CRO uniquely satisfies target coverage across all risk levels while reducing over‑reservation—and thus resource waste—by about half compared to a Chebyshev‑bound baseline.
By Masoud Shokrnezhad, Tarik Taleb
arXiv:2605. 27599v2 Announce Type: replace-cross Abstract: Agentic AI workloads - where a single user goal triggers multi-step orchestration, tool calls, retries, and failure recovery - are being targeted for edge deployment, with NVIDIA, Dell, HP, ASUS, MSI, Acer, and Gigabyte all shipping GB10-based desktop AI systems in 2026.
By Deepak Panigrahy, Aakash Tyagi
AgentPerfBench is a new benchmarking suite designed to evaluate the inference performance of agentic large language models (LLMs) that handle multi‑turn, tool‑using, and context‑expanding tasks. It builds on real traces from agentic benchmarks such as SWE‑Bench and TerminalBench, and generates synthetic profiles that reflect realistic input/output lengths and turn counts. The suite also provides kernel‑level Nsight Compute traces and a multi‑dimensional roofline model to identify hardware bottlenecks and quantify the gap between traditional chat benchmarks and agentic workloads.