arXiv AI By Anjiang Wei, Tianran Sun, Tarun Suresh, Haoze Wu, Ke Wang, Alex Aiken

Quokka: Accelerating Program Verification with LLMs via Invariant Synthesis

Read the original on arXiv AI →

arXiv:2509. 21629v4 Announce Type: replace-cross Abstract: Program verification relies on loop invariants, yet automatically discovering strong invariants remains a long-standing challenge.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning
Sep 10

Robustness of LLM-Generated SystemVerilog Assertions to Semantics-Preserving RTL Transformations

The paper evaluates how robust large language models are at generating SystemVerilog Assertions (SVA) when the underlying RTL code undergoes semantics‑preserving transformations such as operand reordering, identifier renaming, and redundant parenthesization. Using a curated dataset and two open‑source models (Qwen2.5‑Coder‑7B and DeepSeek‑Coder‑V2‑Lite), the authors find that 9.7%–27.0% of behaviors that were correct on the original RTL become incorrect after transformation, revealing significant instability that aggregate accuracy metrics can hide.

By FNU Aditi
arXiv AI
Aug 18

T-LLM Compiler: Trusted LLM-based Code Optimization and Verification Framework

arXiv:2608. 14953v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have opened opportunities to apply high-level code transformations to the field of code optimization, and it has since emerged as one of the most fundamental tasks for LLMs to perform; however, at present, LLMs struggle to apply wide-ranging code optimization tasks due to both the complexity of the code and the inability to independently verify the correctness of the transformations.

By Zahra Fazel, Sunanda Gamage, Shayan Shirahmad Gale Bagi, Amir H. Ashouri, Tomasz S. Czajkowski, Bryan Chan, Reza Azimi, Yaoqing Gao
arXiv AI
Sep 24

Can LLMs Reason About Runtime Behavior? A Repository-Level Dynamic Benchmark

The paper introduces SWE-Flux, a repository‑level benchmark designed to test large language models’ ability to reason about runtime behavior. It contains 480 execution‑grounded instances from 12 real Python repositories, with gold answers automatically harvested from instrumented test executions. Evaluation of five LLMs shows the task remains difficult, with the best model achieving only 37% accuracy, and the benchmark can generate challenging variants through input perturbation.

By Hamed Taherkhani, Mohammad Abdollahi, Melika Sepidband, Hridya Dhulipala, Tien N. Nguyen, Hadi Hemmati