Microsoft Research introduces PDD Kit, a tool that formally verifies code generated by AI agents for full‑stack web applications. By translating developer intent into Verus specifications, PDD Kit proves that the AI‑written code satisfies these specifications for all inputs, supplementing proofs with tests when necessary. This approach allows developers to review high‑level intent rather than inspecting every line of code.
The article discusses the essential skill of effectively instructing coding agents and verifying their changes. It highlights that while line‑by‑line code review is one method, it is not the most efficient way to validate software changes. The focus is on confidently guiding agents and confirming correct implementation without exhaustive inspection.
arXiv:2606. 32007v1 Announce Type: new Abstract: We study agentic code generation in Dafny, where a model must generate both executable code and the proof artifacts for verification.
arXiv:2606. 26649v1 Announce Type: new Abstract: Agent safety in high-stakes domains requires formal policy enforcement, but most existing approaches either rely on probabilistic guardrails (fine-tuned classifiers, prompt-based steering) that offer no formal guarantees, or on hand-coded symbolic enforcement that does not scale to the breadth of real policy specifications.
arXiv:2607. 09217v1 Announce Type: new Abstract: In this system paper, we present OpenProver, an open-source system for LLM-driven automated theorem proving (ATP) with integrated Lean 4 formal verification.
arXiv:2606. 01008v1 Announce Type: cross Abstract: We present a benchmark for evaluating AI models and agents on real-world formal software verification tasks.
The paper investigates the reliability of software produced by agentic AI by comparing AI-generated versions of ten well-known Linux utilities to their human-written counterparts. Using fuzz testing (both black-box and coverage-guided AFL++), the authors find that AI-generated code is often as reliable or more reliable than the latest human versions, with fewer memory errors but a higher incidence of hangs. The study emphasizes that robust AI-generated software requires careful prompting, skilled human oversight, and that the AI workflow can serve as a cost-effective specification for sustainable code.
The paper introduces MAGS, a multi-agent framework that automatically generates executable programs with formal safety guarantees. MAGS translates LLM-generated code into the verification-aware language Dafny, repairs any safety violations using verifier feedback, and then compiles the verified code back into executable form. Evaluations on 220 diverse examples—including CUDA kernels, terminal scripts, and robotic-arm tasks—show a 100% success rate in producing programs that meet frozen safety specifications, with additional safety and functional tests confirming strong performance across domains.
The paper discusses the need for independent assurance when coding agents produce both software and its correctness evidence. It identifies four gaps—requirement, conformance, evaluation, and model—between stakeholder intent and production, and proposes a verification framework that separates building from verifying. The framework introduces a Verification Contract, a specification-first phase, and an implementation-aware phase using mutation analysis, property-based testing, and fuzzing, and is evaluated in a study showing that an external verifier detected more seeded faults than the builder’s own tests.
VeriCUA is a platform-agnostic framework that treats verification of computer-using agents as dynamic evidence gathering and assessment. It constructs a task rubric from available context, then selects GUI or code checks to resolve unmet requirements and combines their findings under explicit evidence rules. Experiments demonstrate that VeriCUA achieves higher agreement with reference labels than baseline methods, especially on hard offline cases and in rejecting false completion claims online.
VeriCUA is a platform-agnostic framework that verifies computer-using agents by treating verification as dynamic evidence gathering and assessment. It constructs a task rubric from available context, then selects GUI or code checks to resolve unmet requirements, combining findings under explicit evidence rules. Experiments demonstrate higher agreement with reference labels than baselines, especially on hard offline cases and in rejecting false completion claims online.
The paper introduces VerifierBench, a benchmark comprising 222 tasks from six agent benchmarks and 4,993 hidden test cases designed to evaluate the generation of executable verifiers by large language model agents. It presents Verifier4Verifier (V4V), a six-check audit that screens source verifiers and test suites, and reports that across 19 models, normalized balanced accuracy ranges from 10.6 to 62.1, with generated verifiers more reliably rejecting invalid solutions than accepting valid ones. Case studies highlight that many false rejections stem from verifiers that accept no tested solution, revealing unsupported assumptions about solution formats, tool use, and code changes that exclude valid solutions, and underscore the need to test both acceptance of correct solutions and other permissible solutions.
The paper introduces Back-to-the-Future (BTTF), an agentic framework designed to automate post-simulation verification and waveform debugging in chip design. BTTF transforms large simulation dumps into a normalized SQLite database and uses a multi-agent orchestration engine to convert natural-language queries into schema-aware SQL, linking anomalies to versioned RTL code. In a benchmark of 150 queries, BTTF achieves 95.33% execution accuracy, demonstrating a viable path toward autonomous EDA verification.