arXiv AI By Xiaonan Xu, Wenjing Wu

Validation Evidence in LLM Repair Agents: How Much of What Passes Actually Tests the Bug?

Read the original on arXiv AI →

arXiv:2607. 28871v1 Announce Type: cross Abstract: When a repair agent runs a test and sees it pass, the result is treated as evidence about the reported defect.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Computation and Language
Aug 31

Why Didn't It Check? Unsupported Final Claims and Their Repair in Two Tool-Equipped Language Models

The study investigates how language models equipped with tools can still produce unsupported final claims, even when a single tool call could resolve the uncertainty. It defines two metrics—occurrence (how often unsupported claims arise) and conditional repair (how often they are fixed when evidence is provided). Experiments on Qwen3-32B and Gemma 4 show that providing the missing evidence consistently repairs all unsupported claims in the Qwen3-32B setup, while the Gemma 4 model never produced unsupported claims under the tested conditions.

By Justin Bronder