Automatic Harness Evolution for Hardware Design Verification: Can LLMs Consolidate Gains Across Discovered Harnesses?

The study investigates whether large language models (LLMs) can automatically improve the harnesses used for hardware design verification. By evolving harnesses around a fixed subject model on 12 proprietary root‑cause localization tasks, the researchers found that automatically evolved harnesses increased completed attempts by 71‑76% and task coverage by 80‑100%, though overall correct attempts improved only 18‑24%. Despite these gains, the evolved harnesses did not consistently consolidate into a single dominant solution across tasks and metrics, and a separate cross‑benchmark case showed that a repair harness could yield a 35.6% increase in functional passes over a baseline.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Machine Learning.

• llms
• agents
• benchmarks