DeFiFlowBench: Benchmarking and Improving Safe Executability in Natural-Language DeFi Workflow Synthesis

DeFiFlowBench is a benchmark comprising 207 natural‑language prompts for synthesizing DeFi workflows, evaluating graph coverage, configuration completeness, and declared safety predicates, and testing trade configurations on a local EVM. The study shows that direct, constrained, and few‑shot prompting still yield unsafe executions, and that a slippage bound derived from a quote does not prevent price impact. The proposed Koan‑Safe system—combining a prompt‑only intent parser, a replaceable generator, and structural repair—achieves a higher static safety proxy score and records no unsafe executions on the benchmark, while ablation studies reveal the limits of default safety thresholds and the need for explicit trade protections. whyItMatters":"The results demonstrate that current prompting methods can still authorize costly trades and that explicit safety mechanisms like Koan‑Safe are necessary to prevent unsafe DeFi workflow executions."

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• benchmarks
• safety