The paper introduces Conformal Reservation for Orchestration (CRO), an online algorithm that uses past execution traces to allocate compute and bandwidth for agentic AI systems at dispatch time. CRO provides finite‑sample, distribution‑free coverage guarantees, ensuring that resource reservations meet specified probabilistic deadlines. Simulations demonstrate that CRO uniquely satisfies target coverage across all risk levels while reducing over‑reservation—and thus resource waste—by about half compared to a Chebyshev‑bound baseline.
arXiv:2606. 11440v1 Announce Type: new Abstract: Existing multi-agent LLM orchestration methods, ranging from brute-force ensembles to learned routers, select models and topologies based on task and model features.
arXiv:2609.17193v1 Announce Type: new Abstract: Large language model (LLM)-powered agentic AI services increasingly demand low-latency inference, motivating the deployment of LLMs across distributed...
The paper introduces agentic-eCAL, an extension of the Energy Cost of AI Lifecycle metric to evaluate multi‑agent AI workflows across the edge‑cloud continuum. By combining a two‑rate energy model with OSI‑layer transport analysis, the authors quantify that inter‑agent text transfer accounts for only 0.25% of total workflow energy, highlighting that the main energy cost lies in additional inference and context processing triggered by communication. The study uses extensive GPU benchmarks on NVIDIA A100/H100 with 16 open‑weight models and 8 orchestration topologies to validate the metric and explore placement implications.
arXiv:2610.07094v1 Announce Type: cross Abstract: LLM deployment is shifting from single-turn completion to agentic trajectories in which a model plans, calls tools, reads results and reasons at test...
The paper introduces a multi-objective reinforcement learning framework for planning AI data center build-outs, treating the long-term infrastructure decisions as a tiered Markov decision process. It proposes a preference-conditioned agent that can generate policies balancing cost, carbon, water, and reliability, and validates the approach on real site data across different commitment regimes. The study also presents a lightweight counterfactual rollout model to assess long-horizon dynamics and releases the associated environment, agents, and evaluation tools.
The paper argues that AI deployment performance depends on interactions among compression, compiler transformations, and serving policies rather than just model architecture. It introduces a three‑layer taxonomy—model‑level techniques, compiler transformations, and system policies—and frames deployment as a constrained multi‑objective optimization problem over accuracy, latency, throughput, memory footprint, and energy. The authors propose an evidence protocol for comparable benchmarking and synthesize data from edge and data‑center platforms to show that cross‑layer interactions drive deployment outcomes, concluding with a constraint‑aware selection procedure and open research problems.
The paper introduces A-LEMS, a framework that measures AI energy consumption at the goal level rather than per model invocation, defining Energy per Successful Goal (EpG) as total workflow energy divided by successfully completed goals. It also proposes the Orchestration Overhead Index (OOI) to quantify the extra energy cost of agentic systems compared to linear execution, revealing significant overheads across various hardware platforms and task families. The study demonstrates that traditional per-inference metrics underestimate the true energy cost of completing user goals in agentic AI workflows.
arXiv:2606. 01162v1 Announce Type: new Abstract: Workflow scheduling in cloud computing demands the intelligent allocation of dynamically arriving, graph-structured workflows with varying deadlines onto ever-changing virtual machine resources.
GAUSS is a workload‑aware analytical and Monte‑Carlo simulator designed to predict the full distributions of time‑to‑first‑token (TTFT) and time‑between‑tokens (TBT) for large language model (LLM) inference serving. It combines calibrated, phase‑specific latency surfaces for prefill and decode with a Lindley‑recurrence queueing model, using ensemble Monte‑Carlo sampling to estimate performance rather than replaying a single trace. This approach enables operators to evaluate workloads, routing policies, and hardware configurations that have never been deployed, and to capture latency tails that trace‑driven simulators typically miss, thereby supporting operating‑point selection, traffic segmentation, and scheduling capacity decisions.
arXiv:2601. 10560v2 Announce Type: replace-cross Abstract: Multi-agent systems (MAS) coordinate multiple LLM-powered agents through structured workflows, gaining reasoning power but incurring high inference latency from multi-step execution and repeated model invocations.
arXiv:2609.06128v1 Announce Type: new Abstract: Production LLM agents execute tool-calling loops, retrieval chains, and compositional workflows in multiple modes, yet execution semantics are often co...
arXiv:2606. 27743v1 Announce Type: cross Abstract: Large Language Models (LLMs) inference is typically deployed under a static resource assumption, where models execute a fixed computational graph regardless of the runtime environment.