FaulT-Bench is a new benchmark comprising 200 network troubleshooting scenarios across eight topologies, designed to test large‑language‑model agents on realistic, noisy tickets that may contain false premises or incorrect fault claims. The benchmark includes 72 rewritten tickets that vary reporter confidence and detail, and evaluates agents via an automated harness that scores diagnoses on outcome, fix, and reasoning quality. Results show that while agents perform well on accurate tickets, they degrade sharply on healthy networks with misleading reports, highlighting the importance of ticket wording over content.
arXiv:2608. 12002v1 Announce Type: new Abstract: Agents are increasingly considered for automating network operations and maintenance, where engineers must diagnose network faults, optimize configurations to enhance services, and reduce operational costs while acting under strict constraints.
By Xingyu Yan, Tingting Dai, Antonio De Domenico, Mohamed Sana, Nicola Piovesan, Changchang Li, Bowen Liu, Kun Jiang, Mengjie Zhang, Dingcheng Shan, Jing-Cheng Pang, Chenwei Wu, Sijie Wu, Lianying Chao, Haoran Cai, Jiantao Ye, Xubin Li, Simon Mark Lucas, Xin Chen
arXiv:2609.14422v1 Announce Type: new
Abstract: Agentic Network Operations (NetOps) are an emerging paradigm promising to enable workload-aware, self-adjustable, and reliable autonomous networks. Whi...
By Tobias Labarta, Frederik Pahde, Novak Boskov, Maximilian Dreyer, David Birkenberger, Manzoor Ahmed Khan, Sebastian Lapuschkin, Wojciech Samek
FaulT-Bench is a new benchmark comprising 200 network troubleshooting scenarios across eight topologies, designed to test large‑language‑model agents on realistic, noisy user tickets that may contain false premises or incorrect fault claims. The benchmark includes 72 rewritten tickets that vary reporter confidence and detail while keeping the network state constant, allowing isolation of the impact of ticket wording on diagnosis. Evaluation of agents such as SADE, ReAct, and Claude Code shows they perform well on accurate tickets but degrade sharply on misleading or healthy‑network tickets, revealing differing failure modes and highlighting the importance of robust reasoning over unreliable input.
By Kuan-Hao Tseng, Niruth Bogahawatta, Yasod Ginige, Kunjan Patel, Kosta Dakic, Suranga Seneviratne
The paper introduces rebuild‑dossier, an open‑source tool that locks an application’s real interface before code is written and enforces one‑test‑at‑a‑time building through automated checks. In experiments, a compliant agent failed a held‑back test while a rule‑breaking agent passed, showing that a passing test suite can be gamed. The study also demonstrates that the automated check mechanism, rather than interface‑locking alone, is crucial for reliable rebuilds, and that multi‑level verification catches errors that single‑level checks miss.
By Parker Fawcett
arXiv:2609.17930v1 Announce Type: new
Abstract: As AI agents take on long, autonomous tasks, we increasingly oversee rather than perform the work, yet we still judge them almost entirely by whether t...
By Salman Rahman, Yubin Kim, Mihir Parmar, A. Ali Heydari, Genglin Liu, Simon A. Lee, Weizhi Zhang, Arian Hosseini, Ahmed A. Metwally, Yuzhe Yang, Baharan Mirzasoleiman, Xin Liu, Pavel Izmailov, Saadia Gabriel, Mark Malhotra, Shwetak Patel, Daniel McDuff, Hamid Palangi
arXiv:2607. 11098v1 Announce Type: cross Abstract: Tool-using LLM agents are mostly evaluated assuming all tools work.
By Aritra Mazumder, Nusrat jahan Lia
arXiv:2608. 03499v1 Announce Type: new Abstract: Recent advances in persistent personal-agent frameworks are making human-centered agent networks realistic deployment targets: each user can be served by an AI agent that acts on the user's behalf, maintains state, and communicates with other agents through social and task relations.
By Prince Zizhuang Wang, Aojie Yuan, Haiyue Zhang, Xiyang Hu, Yue Zhao, Shuli Jiang
The paper introduces EvidenceNet, a runtime assurance layer designed to verify that coordinated AI agent operations achieve an operator’s intended network-wide outcomes across multiple administrative domains. EvidenceNet collects post-change observations from the required authority scopes, checks their freshness and validity, and uses a verifier agent to assess observation content. Experiments on live routing networks demonstrate that this approach can detect successful outcomes that configuration-action logs alone miss, and it rejects completions when observations are sourced incorrectly, substituted, or stale.
By Tianzhu Zhang, Chih-Kai Huang, Meikang Qiu
arXiv:2609.13672v1 Announce Type: new
Abstract: AI agents can be interrupted while editing files, calling tools, or carrying out multi-step tasks. Restarting repeats completed work, but continuing fr...
By Zhihui Zhang, Wei Liu
arXiv:2608. 12761v1 Announce Type: new Abstract: Agentic workflows are commonly evaluated by whether they reach the correct outcome.
By Jesus Salas
arXiv:2607. 16345v1 Announce Type: cross Abstract: Modern agentic systems increasingly rely on skills: installable packages of natural language and code that teach an LLM agent to perform a domain task.
By Tejas Singh Anand, Yuet Ying Christina Wang, Wanting Jiang, Steve Masson, Tian Zheng, Bingjie Zhou