The paper investigates how AI agents that generate smart contracts from plain‑English descriptions can lose intended behavior, a critical issue because smart contracts are immutable once deployed. The authors design an execution‑based evaluation that replays transactions on both the AI‑written contract and the original, measuring agreement and identifying discrepancies. They find that common grading tools miss many mistakes, and propose an elicitation‑first approach where the agent asks a short checklist of questions before coding, which significantly improves correctness across multiple AI models.
By Abhinav Goel, Agostino Capponi, Alfio Gliozzo, Chaitya Shah
arXiv:2606. 26216v1 Announce Type: cross Abstract: We present CyberChainBench, a benchmark for evaluating LLM-based agents on smart contract security across three complementary tasks: vulnerability detection, exploit generation, and patch synthesis.
By Jintao Huang, Fengqing Jiang, Radha Poovendran, Zhiqiang Lin
arXiv:2603. 26270v2 Announce Type: replace-cross Abstract: Smart contracts govern billions of dollars in decentralized finance (DeFi), yet automated vulnerability detection remains challenging because many vulnerabilities are tightly coupled with project-specific business logic.
By Ziqiao Kong, Wanxu Xia, Chong Wang, Yue Xue, Yi Lu, Pan Li, Shaohua Li, Zong Cao, Yang Liu
The paper presents a method that employs Large Language Models to automatically inject known vulnerabilities into Solidity smart contracts. Using a multi-step validation pipeline, the authors generate nearly 1,000 candidate contracts from real-world sources, ultimately confirming 32 vulnerable variants across 25 vulnerability types. These validated contracts are then used to evaluate the coverage of three static analysis tools, highlighting both complementary strengths and gaps in current detection approaches.
By Luca Migliaccio, Roberto Natella, Naghmeh Ivaki, Nuno Laranjeiro, Marco Vieira
AgentXploit is a two‑role auditing system that separates repository‑level attack‑path discovery from runtime exploitation for AI agents. The Analyzer Agent traces attacker‑controlled inputs to sensitive operations and records candidate attack paths, while the Exploiter Agent turns these paths into concrete attacks and refines them using runtime feedback. The system is evaluated on AgentXploit‑Bench, a benchmark of 72 reproducible vulnerabilities across 12 open‑source AI‑agent systems, achieving 59.3% end‑to‑end success compared to 38.4% for Codex, and 79.2% attack success on AgentDojo versus 52.7% for AgentVigil.
By Weida Liang, Shi Qiu, Zhun Wang, Simon Sure, Xiaoyuan Liu, Tianneng Shi, Zhaorun Chen, Wenbo Guo, Dawn Song
arXiv:2608.22103v1 Announce Type: new
Abstract: As agents grow more capable and autonomous, their tendency to reward hack, satisfying a task's checks while violating its intent, becomes an increasing...
By Amit Roth, Ivan Bercovich, Yonathan Efroni