Incident‑Arena is a new benchmark for AI coding agents focused on production incident response, featuring 20 tasks derived from real‑world open‑source software. Each task deploys a production application on an ephemerally created Kubernetes cluster, injects faults at various layers, and applies a sustained load profile. The benchmark introduces functional verifiers that maintain system‑level metrics while ensuring safe repairs, and shows that current frontier models achieve below 64.3% across the tasks, highlighting challenges in diagnosis, repair, and regression safety.
arXiv:2605. 07161v3 Announce Type: replace Abstract: AI agents are increasingly used to diagnose and mitigate failures in production systems, known as agentic Site Reliability Engineering (SRE).
arXiv:2608. 11234v1 Announce Type: new Abstract: Managing modern computing infrastructure has become a steadily harder problem due to the ever-increasing complexity.
arXiv:2602.02475v2 Announce Type: replace Abstract: AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy...
arXiv:2602. 16666v3 Announce Type: replace Abstract: AI agents are increasingly deployed to execute important tasks.
arXiv:2606. 29193v1 Announce Type: cross Abstract: LLM-based agents are reshaping microservice operations into AgentOps, where benchmarks are key to evaluating failure diagnosis over multimodal observability data.
ERPBench introduces a new evaluation paradigm for computer-use agents that operate via screenshots and simulated actions, focusing on enterprise software such as ERP systems. The benchmark tests agents on a live, reproducible ERP platform and scores tasks against ground-truth database values, highlighting challenges like dense interfaces, multi-step interactions, and persistent record errors. Experiments with six agents show that strong general GUI performance does not translate to reliable enterprise outcomes, with many agents frequently saving incorrect data.
arXiv:2605. 27898v2 Announce Type: replace Abstract: As LLMs are increasingly deployed as agents, reliable assessment of their agentic capabilities has become essential.
arXiv:2606. 09122v1 Announce Type: cross Abstract: Cloud network infrastructure at hyperscale presents unique operational challenges where traditional human-driven incident response cannot keep pace with the volume, velocity, and complexity of failures.
arXiv:2606. 19380v3 Announce Type: replace-cross Abstract: Software engineering and deployment are increasingly delegated to AI coding agents.
arXiv:2606. 22678v2 Announce Type: replace-cross Abstract: Agentic coding harnesses - such as Agent-Skills, Superpowers, and Agent-Rigor - are increasingly deployed to augment underlying LLMs for real-world software engineering tasks.
arXiv:2506.03828v4 Announce Type: replace Abstract: AI for Industrial Asset Lifecycle Management aims to automate complex operational workflows, such as condition monitoring and maintenance schedulin...
PANDA is a decentralized architecture for large-scale, fault-tolerant multi-agent systems that enables heterogeneous agents to discover each other's capabilities and self-organize into specialized teams for each task. It decouples collective communication from team communication, allowing agents to participate in multiple teams simultaneously and load-balance tasks across the collective. PANDA supports three planning and execution patterns—star, chain, and mesh—detects and recovers from infrastructure and orchestration failures, and uses a web-of-trust model for governance without a central bottleneck.