arXiv:2606. 13706v1 Announce Type: cross Abstract: We present HierSVA, an integrated suite that combines a pipeline, dataset, and benchmark for LLM-driven hierarchical hardware formal verification.
The paper evaluates how robust large language models are at generating SystemVerilog Assertions (SVA) when the underlying RTL code undergoes semantics‑preserving transformations such as operand reordering, identifier renaming, and redundant parenthesization. Using a curated dataset and two open‑source models (Qwen2.5‑Coder‑7B and DeepSeek‑Coder‑V2‑Lite), the authors find that 9.7%–27.0% of behaviors that were correct on the original RTL become incorrect after transformation, revealing significant instability that aggregate accuracy metrics can hide.
arXiv:2608.21962v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to generate register-transfer-level (RTL) designs from natural-language specifications. However, a...
Spec‑Harness evaluates how well large language models (LLMs) synthesize Java Modeling Language (JML) specifications by measuring behavioral adequacy across precondition and postcondition correctness and completeness. The study shows that while prompt optimization can raise verifier pass rates, many accepted specifications remain behaviorally weak, either over‑ or under‑constraining inputs and outputs. Spec‑Harness also serves as a feedback mechanism that improves the quality of specifications generated by general‑purpose coding agents and a specialized JML agent.
arXiv:2603. 15510v2 Announce Type: replace Abstract: The synthesis of inductive loop invariants remains a critical bottleneck in automated program verification.
arXiv:2607. 28877v1 Announce Type: cross Abstract: Verification consumes the majority of modern chip design effort, yet the formal verification tools that provide mathematical guarantees of correctness remain expensive and restrictively licensed.
arXiv:2609.35841v1 Announce Type: cross Abstract: Mutation testing evaluates test-suite adequacy by injecting synthetic faults into program code. However, traditional rule-based tools often generate...
arXiv:2609.39568v1 Announce Type: cross Abstract: Large language models (LLMs) may generate unreliable code on corner cases missed by testing, while formal verification can provide machine-checkable...
arXiv:2602. 09464v2 Announce Type: replace-cross Abstract: Vericoding refers to the generation of formally verified code from rigorous specifications.
arXiv:2609.21190v1 Announce Type: cross Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corr...
EvoUndo is a framework that enables large language model agents to self‑evolve—modifying prompts, tools, and execution harnesses—while ensuring that these changes can be reliably reversed across different states. The study evaluates EvoUndo on 600 unseen one‑shot self‑evolution tasks, finding that 197 capability‑improving mutations fail recoverability checks. By extending the recovery language and adding exact state‑address diagnostics, the framework recovers up to 191 out of 197 failures, demonstrating that robust self‑evolution requires co‑designing verification, grounding, witness semantics, and recovery expressivity.
arXiv:2607. 23425v1 Announce Type: cross Abstract: Large language models increasingly write TLA$^{+}$ formal specifications from natural-language descriptions, but progress is hard to measure: existing resources grade by resemblance to a reference or by whether the output parses, neither of which shows correctness.