arXiv:2608. 12002v1 Announce Type: new Abstract: Agents are increasingly considered for automating network operations and maintenance, where engineers must diagnose network faults, optimize configurations to enhance services, and reduce operational costs while acting under strict constraints.
arXiv:2511. 00651v2 Announce Type: replace Abstract: Telecom networks are rapidly growing in scale and complexity, making effective management, operation, and optimization increasingly challenging.
FaulT-Bench is a new benchmark comprising 200 network troubleshooting scenarios across eight topologies, designed to test large‑language‑model agents on realistic, noisy tickets that may contain false premises or incorrect fault claims. The benchmark includes 72 rewritten tickets that vary reporter confidence and detail, and evaluates agents via an automated harness that scores diagnoses on outcome, fix, and reasoning quality. Results show that while agents perform well on accurate tickets, they degrade sharply on healthy networks with misleading reports, highlighting the importance of ticket wording over content.
FaulT-Bench is a new benchmark comprising 200 network troubleshooting scenarios across eight topologies, designed to test large‑language‑model agents on realistic, noisy user tickets that may contain false premises or incorrect fault claims. The benchmark includes 72 rewritten tickets that vary reporter confidence and detail while keeping the network state constant, allowing isolation of the impact of ticket wording on diagnosis. Evaluation of agents such as SADE, ReAct, and Claude Code shows they perform well on accurate tickets but degrade sharply on misleading or healthy‑network tickets, revealing differing failure modes and highlighting the importance of robust reasoning over unreliable input.
The paper introduces NetArtifactBench, a benchmark designed to evaluate whether AI agents can detect and repair inconsistencies in network experiment records while preserving supported claims. It tests 23 agent configurations on 52 instances with injected inconsistencies, finding an average pass rate of 65.3 % but no runtime exceeding 30 % for complex repairs that require recovering implicit relations and propagating changes across artifacts. The results highlight a clear distinction between local corrections and full record-level repair, leading the authors to argue that artifact integrity should be a primary design and evaluation criterion for AI agents in network systems.
arXiv:2605. 08717v2 Announce Type: replace-cross Abstract: Software engineering agents are increasingly deployed in evaluable engineering environments, yet post-failure recovery remains costly, manual, and ad hoc.
arXiv:2602.02475v2 Announce Type: replace Abstract: AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy...
arXiv:2607. 13548v1 Announce Type: new Abstract: Identifying root causes in production microservice failures requires reasoning over large-scale, multimodal telemetry spanning metrics, logs, and traces, a problem that has proved resistant to both classical and LLM-based approaches.
arXiv:2608. 05263v1 Announce Type: new Abstract: Multi-agent orchestration frameworks are moving from demos to production, yet benchmarks typically report task accuracy without diagnosing why a pipeline failed, where a cascade began, or which routing decision caused the breakdown.
ParaRecover is a new process-level benchmark designed to evaluate error localization and recovery in multi-turn parallel tool-use agents. It contains 10,626 instances across two difficulty levels, built on a taxonomy of 14 error types that cover planning dependencies, tool selection, and argument matching. The benchmark introduces the SDE rubric, which assesses structural integrity, diagnostic reasoning, and evolutionary strategy during agent execution, and demonstrates that it can guide improvements in agents’ reflective recovery capabilities.
Identifying root causes in production microservice failures requires reasoning over large-scale, multimodal telemetry spanning metrics, logs, and traces, a problem that has proved resistant to both classical and LLM-based approaches. The OpenRCA dataset exemplifies these challenges: it is large-scale, multimodal, and lacks detailed domain knowledge, and yields consistently low accuracy across all existing methods.
arXiv:2609.14422v1 Announce Type: new Abstract: Agentic Network Operations (NetOps) are an emerging paradigm promising to enable workload-aware, self-adjustable, and reliable autonomous networks. Whi...
arXiv:2607. 18754v1 Announce Type: new Abstract: LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it.