← Back to all news
arXiv Computation and Language By Gabriel Dautovi, Jie JW Wu

VeeReview: Measuring Build-Verify Pairs in Agent-Authored Pull Requests from a V-Model Perspective

Read the original on arXiv Computation and Language →

The paper introduces VeeReview, a tool that classifies build‑and‑verify pairs in pull requests using repository signals and a language‑model judge. Evaluated on 1,454 agent‑ and human‑authored pull requests, VeeReview found lower build‑only rates for agent‑authored changes but failed to meet calibration and agreement thresholds. The study concludes that a mechanically decided component and validated human reference are essential for reliable verification‑practice comparisons.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv Computation and Language.

Related stories

arXiv AI
Oct 16

CodeHealth Predicts the Token Cost of Agentic Code Review: A Pre-Registered Study of 465 Agent-Authored Pull Requests

The study examines whether the CodeHealth metric can predict the token cost of agentic code review and fix for pull requests. Using a pre‑registered observational design, 465 agent‑authored PRs were processed by a five‑agent pipeline across three LLMs, and mixed models showed that each CodeHealth point corresponded to a 3.5%–6.1% reduction in token usage, independent of PR footprint. The effect was strongest during the review stage and appears mediated by file size rather than PR size.

By Lo Gullstrand Heander, Markus Borg, Emma S\"oderberg, Christofer Rydenf\"alt
llmsagents
More like this →
arXiv AI
Oct 2

VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks

VeriHarness is a method that enhances verification for large language model agents tackling long‑horizon tasks without needing reference answers at test time. It transforms the base LLM into an agentic verifier by providing a workspace, evidence tools, and reusable verification skills, using disagreement resolution and consensus challenge to evaluate competing claims. Across five benchmarks and two frontier models, VeriHarness outperforms baselines, achieving significant performance gains and demonstrating self‑improvement of verification skills from failure feedback.

By Caiqi Zhang, Rujun Han, Zifeng Wang, Zoey CuiZhu, Nigel Collier, Tomas Pfister, Chen-Yu Lee
llmsagentsbenchmarks
More like this →