SolBehave is a benchmark that evaluates the behavioural correctness of smart contracts generated by large language models, going beyond simple compilation checks. It applies three tests—differential fuzzing, historical‑transaction replay, and DCR‑guided synthetic‑transaction replay—on 250 real Ethereum contracts to assess whether generated code matches the original’s behaviour. The study finds that while compilation success varies widely, only a minority of compiled contracts are behaviourally equivalent, highlighting that compilation alone is insufficient for correctness.
By Giovanni Pio Delvecchio, Stefano Fantazzini, Giacomo Frisoni, Gianluca Moro, Lorenzo Bacchiani, Mario Bravetti, Thomas Troels Hildebrandt
arXiv:2602. 09464v2 Announce Type: replace-cross Abstract: Vericoding refers to the generation of formally verified code from rigorous specifications.
By Haoyu Zhao, Ziran Yang, Jiawei Li, Deyuan He, Zenan Li, Chi Jin, Venugopal V. Veeravalli, Aarti Gupta, Sanjeev Arora
arXiv:2608. 12599v1 Announce Type: new Abstract: Multi-turn dialogues let users revoke constraints as easily as impose them, but revocation does not reliably take effect: models keep enacting withdrawn requirements (occasionally beneath comments asserting their removal), a failure we call \emph{behavioral relapse}, or revocation inertia.
By Haoyuan Zhu
arXiv:2610.17358v1 Announce Type: cross
Abstract: Formal verification proves that a program satisfies its specification, but says nothing about whether the specification captures what the author mean...
By Martin Amster, One An, Holly Baker, Fernanda Graciolli, Simon Henniger, Nada Amin
arXiv:2609.21190v1 Announce Type: cross
Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...
By George Ma, Benjamin Mikek, Haoyu Li, Ferhat Erata, Yuhao Zhang, Zeren Shui, Behrooz Omidvar Tehrani, Jun Huan, Murali Krishna Ramanathan, Somayeh Sojoudi, Hao Zhou, Anoop Deoras
NOMOS is a four‑pass compiler that transforms natural‑language policies into deterministic, statically verified tool‑call gates for large‑language‑model agents. By performing tool‑schema‑level checks without relying on provers, solvers, or additional LLM calls, it repairs or rejects 37% of airline and 13% of retail policy candidates, enabling most shipped rules to become operable. In benchmark tests, the compiled gates reduce policy violations from 66.3% to 2.6% in airline scenarios and from 30.8% to 6.9% in retail, while achieving a zero attack success rate on banking and low rates on other suites, all with microsecond decision times.
By Min-Young Yu, Tony Kim, Jang Won Choi