arXiv AI
Sep 10

HoarePrompt: Structural Reasoning About Program Correctness in Natural Language

HoarePrompt is a new method that applies program verification concepts to natural language requirements, using large language models to generate step‑by‑step natural language descriptions of program states. It incorporates a few‑shot k‑induction technique to handle loops and then evaluates whether the annotated program satisfies the requirements. On the CoCoClaNeL dataset, HoarePrompt raises the Matthews correlation coefficient by 61% over zero‑shot chain‑of‑thought prompts and by 106% over test‑generation classifiers, with the inductive reasoning component adding a 26% MCC improvement.

By Dimitrios Stamatios Bouras, Yihan Dai, Tairan Wang, Yingfei Xiong, Sergey Mechtaev