arXiv AI

Can AI Agents Deliver Verifiable Network-Wide Outcomes Across Authority Boundaries?

The paper introduces EvidenceNet, a runtime assurance layer designed to verify that coordinated AI agent operations achieve an operator’s intended network-wide outcomes across multiple administrative domains. EvidenceNet collects post-change observations from the required authority scopes, checks their freshness and validity, and uses a verifier agent to assess observation content. Experiments on live routing networks demonstrate that this approach can detect successful outcomes that configuration-action logs alone miss, and it rejects completions when observations are sourced incorrectly, substituted, or stale.

arXiv AI
Aug 24

Testing and Evaluation of Agentic AI Systems In Military Command and Control

The paper examines how agentic AI systems intended for military command and control are tested and evaluated. It reviews 240 testing practices across eight dimensions and three lifecycle stages, uncovering eight assumptions—grouped into system specifiability, stability, composability, and supervisability—whose validity is weakened by agentic properties. Consequently, test results may meet procedural standards but do not guarantee that fielded behavior matches tested behavior, leading the authors to propose ten assurance claims and suggest that uncertainty be managed through deployment‑time monitoring and defined expiry conditions.

By Ulysse Richard, Heather Frase, Sarah Cao, Di Cooke, Sebastian Kwon, Adrianna Tan
arXiv AI
Oct 1

Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents

The paper introduces Runtime Assurance Contracts (RAC) as a formal policy framework for high‑risk AI agents, addressing the "assurance‑transition gap" by binding autonomy boundaries, component eligibility, evidence state, transition policy, human‑review capacity, and non‑compensatory gates. RAC allows soft metrics to influence routing while mandating retries, switches, escalations, deferrals, or stops when mandatory gates fail or are unknown, ensuring aggregate performance cannot alone authorize action. The authors define the contract, evidence record, permission rule, and five invariants, and evaluate RAC through deterministic failure‑injection studies, hand‑authored traces, and a prospective synthetic holdout, comparing it to score‑only and restricted protocol baselines.

By Serhii Zabolotnii
arXiv AI
Aug 20

One Gate Is Not Enough: Composing Stateful Pre-Action Controls for Agentic AI

The paper investigates how multiple pre‑action controls—authority, resource, and evidence gates—interact in agentic AI systems. It formalizes remediation‑induced control coupling, showing that remediation can invalidate earlier judgments and that the order of remediation matters. The authors propose a remediate‑and‑regate protocol to restore soundness, analyze non‑commuting remediation operators, and demonstrate the approach on a deterministic open‑data artifact with three published engines.

By Gaston Besanson
arXiv AI
1d ago

Assurance-Conditioned Continuous Authorization for Autonomous AI Agents

The paper introduces Assurance-Conditioned Continuous Authorization (ACCA), a framework that maintains operational authority for autonomous AI agents by separating evidence validity from contextual applicability. ACCA derives assurance claims from eligible evidence, resolves a multidimensional effective-authority envelope, and uses a typed dependency graph for selective re‑evaluation of authorization‑relevant changes. The authors demonstrate the approach with controlled scenario tests in a trading domain, showing implementation‑level evidence for the method’s effectiveness.

By Mehdi Shajari
arXiv AI
Sep 11

Can AI Agents Detect and Repair Artifact Drift in Network Experiments?

The paper introduces NetArtifactBench, a benchmark designed to evaluate whether AI agents can detect and repair inconsistencies in network experiment records while preserving supported claims. It tests 23 agent configurations on 52 instances with injected inconsistencies, finding an average pass rate of 65.3 % but no runtime exceeding 30 % for complex repairs that require recovering implicit relations and propagating changes across artifacts. The results highlight a clear distinction between local corrections and full record-level repair, leading the authors to argue that artifact integrity should be a primary design and evaluation criterion for AI agents in network systems.

By Tianzhu Zhang, Weichen Tao, Changgang Zheng, Yusheng Zheng, Long Chen, Xiaoyi Fan, Meikang Qiu
Hugging Face Trending Papers
Jul 2

Criticality-Based Guard Rail Validation for AI Agent Decisions in Autonomous Telecom Networks

The evolution toward fully autonomous telecommunications networks (Autonomous Network Levels 4-5) requires AI/ML agents to make real-time network decisions without human intervention. However, no standardized runtime mechanism exists to intercept and validate individual inference outputs before they trigger live network state changes, creating risks of erroneous autonomous decisions.

arXiv AI
Jun 17

Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety

arXiv:2605. 12729v2 Announce Type: replace-cross Abstract: Large language models are increasingly being used to support network operations (NetOps) and artificial intelligence for IT operations (AIOps), including incident investigation, root-cause analysis, configuration synthesis, and limited self-healing.

By Muhammad Bilal, Jon Crowcroft, Ruizhi Wang, Xiaolong Xu, Schahram Dustdar
arXiv AI
Oct 21

Runtime Control for AI Agents: Closing the Evaluation--Execution Gap in Operational Workflows

The paper introduces a runtime assurance layer for language‑model agents operating in operational workflows, addressing the evaluation–execution gap where approvals may become invalid after changes in facts, evidence, policy, or authority. The proposed state‑bound configuration separates probabilistic measurement from deterministic adjudication, binds execution allowances to the basis of an approval, invalidates them upon logged changes, and rechecks admissibility at dispatch. In a scripted 42‑case replay, the approach identified 15 later‑invalidated approvals that a non‑re‑evaluating comparator missed, holding 14 affected actions before dispatch and compensating one landed effect.

By Jayesh Bhadane, Ramachandra Yerramsetti, Rajesh Gupta, Sumit Kumar
arXiv AI
Sep 25

When Agents Act Unwatched: The Reduced-Supervision Paradox in Agentic AI

The paper "When Agents Act Unwatched: The Reduced‑Supervision Paradox in Agentic AI" discusses how the promise that AI systems will continue acting after users stop watching creates an accountability inversion. It argues that as stepwise supervision recedes, verification shifts into the runtime infrastructure—authority, records, interrupts, outcome checks, and repair—forming what the authors call the reduced‑supervision paradox. A 63‑artifact audit across research papers and engineering sources shows that agents’ action surfaces are more visible than the mechanisms needed to hold them accountable, with tool mediation and monitoring traces appearing in 40 and 37 artifacts, while checkpoint placement, validator independence, recovery, and contestability are rarely visible. "whyItMatters":"The study highlights that observable action paths can replace accountability when verification is moved onto users after meaningful intervention is no longer possible."

By Hanjing Shi, Dominic DiFranzo