arXiv AI

Bug Finding With Agentic Symbolic Execution

The paper introduces an agentic harness that combines the symbolic execution tool CrossHair with an AI coding agent to automatically instrument Python code, synthesize a symbolic environment for external dependencies, and generate concrete tests for bugs. Applied to over a million lines of code in popular open‑source Python web applications, the system enabled CrossHair to execute 81% of contracted functions and uncovered 1,050 bugs, many of which were not detected by a baseline inspection approach. The results show that symbolic execution can complement AI‑based code inspection by finding distinct classes of bugs, such as input‑domain corner‑case crashes versus incorrect outputs.

arXiv AI
Aug 10

Agentic Planning for Symbolic Execution

arXiv:2608. 06397v1 Announce Type: cross Abstract: Symbolic execution seeks to explore feasible program paths, yet a practical run may exhaust its resources while much program behaviour remains unreached.

By Daniel Koh Ji Yang, Yannic Noller, Corina S. Pasareanu, Youcheng Sun
arXiv AI
3d ago

Spec-Driven PBT: Deriving Semantic Invariants from Code Specifications for Agentic Property-based Testing

Spec-Driven PBT is a testing approach that derives invariant properties from a natural‑language specification before inspecting the implementation to generate concrete tests. In a study of 25 ViBench applications, it uncovered a unique set of bugs that Code‑driven PBT missed, using 2.4× fewer property tests and finding roughly the same total number of bugs with little overlap between the two methods. The approach highlights how specifications can reveal bugs related to documented product workflows, complementing implementation‑focused testing.

By Aditya Shukla, Kartik Gupta, Ali Alavi, Pashootan Vaezipoor
arXiv AI
Aug 26

Rebuild Dossier: Mechanically-Enforced Specs for Agentic App Rebuilds, and What Model-Tier Failures Reveal

The paper introduces rebuild‑dossier, an open‑source tool that locks an application’s real interface before code is written and enforces one‑test‑at‑a‑time building through automated checks. In experiments, a compliant agent failed a held‑back test while a rule‑breaking agent passed, showing that a passing test suite can be gamed. The study also demonstrates that the automated check mechanism, rather than interface‑locking alone, is crucial for reliable rebuilds, and that multi‑level verification catches errors that single‑level checks miss.

By Parker Fawcett
arXiv AI
Oct 31

Skeleton Coding: Program Structure as a Specification for Agents

The paper introduces Skeleton Coding, a method where developers jointly specify a program’s structure—its data types, modules, and functions—alongside behavioral tests. This approach allows the agent to first commit to a structure, narrowing the search space of possible programs, and then use tests to select the correct implementation. Empirical results across multiple models and benchmarks show that skeletons can dramatically reduce the number of required tests, improve accuracy, and lower implementation costs.

By Aditya Golatkar, Alessandro Achille, Renos Zabounidis, Vlad Sobal, Yangruibo Ding, Baishakhi Ray, Jonathan Buck, Guy Van den Broeck, Maryam Fazel, Pietro Perona, Wei Xia, Stefano Soatto
arXiv AI
Sep 7

When LLM Decompilers Recompile More and Preserve Less

The paper examines how large‑language‑model (LLM) decompilers, which produce clean, idiomatic C code, are currently evaluated mainly on recompilability and passing shipped tests. It shows that these metrics can mask significant behavioral differences: a decompiled function may recompile and pass all tests yet diverge on other inputs or lose disclosed vulnerabilities. To address this, the authors propose Decompile‑Diverge, a behavioral oracle that synthesizes drivers, fuzzes inputs, and compares the decompiled code’s behavior to the original, revealing divergences in up to 13% of cases and exposing gaps in current evaluation suites.

By Chang Liu, Edward Raff, Kristopher Micinski