The paper examines how agentic AI systems intended for military command and control are tested and evaluated. It reviews 240 testing practices across eight dimensions and three lifecycle stages, uncovering eight assumptions—grouped into system specifiability, stability, composability, and supervisability—whose validity is weakened by agentic properties. Consequently, test results may meet procedural standards but do not guarantee that fielded behavior matches tested behavior, leading the authors to propose ten assurance claims and suggest that uncertainty be managed through deployment‑time monitoring and defined expiry conditions.
By Ulysse Richard, Heather Frase, Sarah Cao, Di Cooke, Sebastian Kwon, Adrianna Tan
The paper investigates how multiple pre‑action controls—authority, resource, and evidence gates—interact in agentic AI systems. It formalizes remediation‑induced control coupling, showing that remediation can invalidate earlier judgments and that the order of remediation matters. The authors propose a remediate‑and‑regate protocol to restore soundness, analyze non‑commuting remediation operators, and demonstrate the approach on a deterministic open‑data artifact with three published engines.
By Gaston Besanson
The paper introduces the Distinguish-or-Homogenize principle, where an agent can either spend resources to differentiate between latent fault models or alter the system state so that the remaining models share a common acceptable policy, eliminating further diagnosis. This leads to the Last-Chance Policy Identification (LCPI) framework, which evaluates correctness at the reached state rather than the initial one, and defines the Last Identifiable Margin (LIM) as the boundary between distinguishing and homogenizing. For deterministic diagnostic graphs, an Exact-LIM recursion is provided, while for noisy finite-horizon recovery the authors propose Risk-Budgeted Compatibility Planning (RBCP), which searches a compatibility-aware frontier under a hard worst-case failure constraint, demonstrating improved risk-feasible recovery in microservice and MiniGrid scenarios.
By Yibo Guo, Xiaodan Wang
arXiv:2607. 03386v1 Announce Type: new Abstract: Agentic AI systems are increasingly used to edit, refine, and repair decision policies, but evaluating these edits is difficult when per-state expert action labels are unavailable.
By Peiying Zhu, Sidi Chang
The paper introduces rebuild‑dossier, an open‑source tool that locks an application’s real interface before code is written and enforces one‑test‑at‑a‑time building through automated checks. In experiments, a compliant agent failed a held‑back test while a rule‑breaking agent passed, showing that a passing test suite can be gamed. The study also demonstrates that the automated check mechanism, rather than interface‑locking alone, is crucial for reliable rebuilds, and that multi‑level verification catches errors that single‑level checks miss.
By Parker Fawcett
arXiv:2607. 00269v1 Announce Type: new Abstract: LLMs, solvers, and agent teams increasingly generate workflow actions, repairs, and plans, but a generated action may be syntactically valid yet stale, infeasible, conflicting, or destructive of the evidence that triggered a repair.
By Edward Y. Chang, Longling Geng, Emily J. Chang
The paper proposes a claim‑specific verification audit for modular agents that replaces aggregate task scores with evidence‑based evaluations. Each agent conclusion is recorded with supporting evidence and classified as supported, unsupported, unresolved, or not evaluated, along with the boundary of validity. The audit employs three tools—oracle policies, perfect component replacements, and verifier‑score tests—to trace value changes, locate lost value, and assess verifier effectiveness, demonstrated on a portfolio‑allocation agent in a synthetic market.
By Ali Atiah Alzahrani
The paper introduces Runtime Assurance Contracts (RAC) as a formal policy framework for high‑risk AI agents, addressing the "assurance‑transition gap" by binding autonomy boundaries, component eligibility, evidence state, transition policy, human‑review capacity, and non‑compensatory gates. RAC allows soft metrics to influence routing while mandating retries, switches, escalations, deferrals, or stops when mandatory gates fail or are unknown, ensuring aggregate performance cannot alone authorize action. The authors define the contract, evidence record, permission rule, and five invariants, and evaluate RAC through deterministic failure‑injection studies, hand‑authored traces, and a prospective synthetic holdout, comparing it to score‑only and restricted protocol baselines.
By Serhii Zabolotnii
The paper presents a method for turning expert diagnoses of verification failures into reusable guidance for coding agents. By combining executable language definitions in the K framework with a set of procedures for constructing specifications, repairing proofs, and auditing their adequacy, the authors achieve a 164/164 success rate on the HumanEval benchmark after two targeted repairs. They further demonstrate that audits can detect defects missed by successful proofs and evaluate the approach on KleverBench and Optimism proofs, highlighting both progress and remaining challenges.
By Yuqing Zhai, Xiaohong Chen, Lingming Zhang, Sriram Vishwanath, Grigore Rosu
The paper introduces the concept of Compositional Policy Violations (CPVs), where each step in an agentic AI workflow passes its individual compliance check, yet the overall execution violates higher‑level policies such as referral thresholds or authority limits. It categorizes CPVs into four types—Authority Creep, Threshold Laundering, Cumulative Sum Violation, and Context Collapse—and argues that the appropriate remedy depends on where the guarded quantity changes. To address this, the authors propose a provenance‑aware runtime architecture that evaluates policies over complete execution traces, recomputing guarded quantities from raw provenance rather than relying on step‑level outputs.
By Ashwini Kurady, Sri Sai Charith Grandhi, Rajesh Gupta, Sumit Mamoria
The paper investigates how an attacker could manipulate an AI adviser to issue harmful quantum error‑correction updates. It identifies an ambiguity in passive syndrome records that can mislead recovery selection and demonstrates that additional calibration measurements can provide the missing sign information needed for safe updates. By introducing a separate evaluator that only accepts updates when calibration uncertainty and drift bounds certify improvement, the authors show through simulations and surface‑code experiments that harmful proposals are rejected while beneficial ones are retained.
By A. Bar{\i}\c{s} \"Ozg\"uler
arXiv:2609.13672v1 Announce Type: new
Abstract: AI agents can be interrupted while editing files, calling tools, or carrying out multi-step tasks. Restarting repeats completed work, but continuing fr...
By Zhihui Zhang, Wei Liu