← Back to all news
arXiv AI By Giovanni Pio Delvecchio, Stefano Fantazzini, Giacomo Frisoni, Gianluca Moro, Lorenzo Bacchiani, Mario Bravetti, Thomas Troels Hildebrandt

SolBehave: Evaluating the Behavioural Correctness of LLM-Generated Smart Contracts

Read the original on arXiv AI →

SolBehave is a benchmark that evaluates the behavioural correctness of smart contracts generated by large language models, going beyond simple compilation checks. It applies three tests—differential fuzzing, historical‑transaction replay, and DCR‑guided synthetic‑transaction replay—on 250 real Ethereum contracts to assess whether generated code matches the original’s behaviour. The study finds that while compilation success varies widely, only a minority of compiled contracts are behaviourally equivalent, highlighting that compilation alone is insufficient for correctness.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 7

When LLM Decompilers Recompile More and Preserve Less

The paper examines how large‑language‑model (LLM) decompilers, which produce clean, idiomatic C code, are currently evaluated mainly on recompilability and passing shipped tests. It shows that these metrics can mask significant behavioral differences: a decompiled function may recompile and pass all tests yet diverge on other inputs or lose disclosed vulnerabilities. To address this, the authors propose Decompile‑Diverge, a behavioral oracle that synthesizes drivers, fuzzes inputs, and compares the decompiled code’s behavior to the original, revealing divergences in up to 13% of cases and exposing gaps in current evaluation suites.

By Chang Liu, Edward Raff, Kristopher Micinski
llms
More like this →
arXiv AI
Jul 23

Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixes

arXiv:2607. 19843v1 Announce Type: cross Abstract: Large language models (LLMs) have made automated program repair (APR) increasingly practical for real-world bugs, but repairing directly from bug reports remains underconstrained.

By Yuhao Tan, Zhibang Yang, Fangkai Yang, Yuan Yao, Yu Kang, Lu Wang, Pu Zhao, Xin Zhang, Xiaoxing Ma, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
llmsbenchmarks
More like this →
arXiv AI
Sep 21

SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?

arXiv:2609.21190v1 Announce Type: cross Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...

By George Ma, Benjamin Mikek, Haoyu Li, Ferhat Erata, Yuhao Zhang, Zeren Shui, Behrooz Omidvar Tehrani, Jun Huan, Murali Krishna Ramanathan, Somayeh Sojoudi, Hao Zhou, Anoop Deoras
llmsagentsbenchmarkssafety
More like this →