Atlas is a framework that optimizes the deployment of compound AI workflows on heterogeneous clusters by selecting execution plans that satisfy service level objectives (SLOs). It introduces MAP, a Markovian Accuracy Predictor, which estimates configuration accuracy using local conditional accuracy transitions between adjacent workflow stages, avoiding exhaustive end‑to‑end profiling. Atlas formulates plan selection as a mixed‑integer linear program, achieving near‑oracle accuracy while reducing deployment cost by up to 42% and profiling cost by up to 2.6×.
arXiv:2606. 14350v1 Announce Type: cross Abstract: Artificial Intelligence (AI) systems must typically satisfy service-level objectives including accuracy, latency, and cost.
arXiv:2606. 14356v1 Announce Type: cross Abstract: Applications in the 3D Computing Continuum, which unifies edge, cloud, and space, require combining multiple AI tasks such as object detection, time-series analytics, and natural language processing into Compound AI systems.
The paper argues that AI deployment performance depends on interactions among compression, compiler transformations, and serving policies rather than just model architecture. It introduces a three‑layer taxonomy—model‑level techniques, compiler transformations, and system policies—and frames deployment as a constrained multi‑objective optimization problem over accuracy, latency, throughput, memory footprint, and energy. The authors propose an evidence protocol for comparable benchmarking and synthesize data from edge and data‑center platforms to show that cross‑layer interactions drive deployment outcomes, concluding with a constraint‑aware selection procedure and open research problems.
HoliBench is a modular benchmarking and deployment toolkit that jointly measures accuracy, latency, and energy for foundation models across a wide range of devices, from single-board computers to GPU servers. It provides a platform abstraction layer that calibrates cross-device measurements and supports multiple model modalities, inference engines, and quantization levels. Using HoliBench, the authors evaluated 20 models on 7 device types, revealing tradeoffs such as limited latency gains from quantization on low‑precision hardware and diminishing accuracy returns relative to energy consumption, while also showing that single-model profiles can predict multi-model pipeline performance within a few percent.
arXiv:2608. 03311v1 Announce Type: cross Abstract: Enterprise compliance management requires rapid adaptation to evolving regulatory frameworks (e.
arXiv:2601. 20408v2 Announce Type: replace-cross Abstract: Enterprise LLM deployment faces a critical scalability challenge: organizations must optimize models systematically to scale AI initiatives within constrained compute budgets, yet the specialized expertise required for manual optimization remains a niche and scarce skillset.
arXiv:2609.01045v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities as powerful components in agentic systems, enabling sophisticated reasoning and...
arXiv:2608. 00107v1 Announce Type: new Abstract: Agentic systems must repeatedly decide whether to answer directly, decompose a task, invoke a tool, execute code, delegate to a specialist, verify an intermediate result, or recover from failure.
arXiv:2608. 08528v1 Announce Type: new Abstract: Enterprise AI coding assistants incur substantial inference spend, and naive token-cost minimization often fails to reduce end-to-end cost once retries, escalations, and developer wait time are included.
AgentOrch is a benchmark that compares global orchestration—where a complete workflow is generated upfront—with step‑by‑step interaction on 650 enterprise standard operating procedures and 3,220 runtime variants. Across all evaluated models, step‑by‑step remains more accurate, but the performance gap has narrowed by about 79% from early to recent models. The study identifies three methods—incremental workflow construction, revising workflows with execution feedback, and targeted training on complete workflows—that can further close the gap, improving global orchestration accuracy by up to 12.18 percentage points.
arXiv:2610.16867v1 Announce Type: cross Abstract: Reliable agentic tool use requires training data that pair diverse multi-step workflows with executable evidence that the requested outcomes were ach...
arXiv:2604. 05150v2 Announce Type: replace-cross Abstract: We study compiled AI, a paradigm in which large language models generate executable code artifacts during a compilation phase, after which workflows execute deterministically without further model invocation.