arXiv AI
ExecCritic: Learn to Test, Test to Improve for Coding Agents
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 21
AEVAL: From Anecdotal to Deterministic Testing for Agentic Skill Workflows
arXiv:2607. 16345v1 Announce Type: cross Abstract: Modern agentic systems increasingly rely on skills: installable packages of natural language and code that teach an LLM agent to perform a domain task.
arXiv AI
Aug 3
Validation Evidence in LLM Repair Agents: How Much of What Passes Actually Tests the Bug?
arXiv:2607. 28871v1 Announce Type: cross Abstract: When a repair agent runs a test and sees it pass, the result is treated as evidence about the reported defect.
arXiv Machine Learning
Jul 10
TTHE: Test-Time Harness Evolution
arXiv:2607. 08124v1 Announce Type: cross Abstract: The behavior of an LLM agent is determined not only by the underlying model, but also by its harness: the executable program that constructs context, invokes tools, verifies intermediate results, and recovers from failures.
arXiv AI
Aug 11
FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents
arXiv:2608. 08570v1 Announce Type: new Abstract: Rejection sampling fine-tuning (RFT) is widely used to train code agents by generating trajectories on verifiable software engineering tasks, retaining those that pass the tests, and fine-tuning on the successful rollouts.
arXiv Computation and Language
Aug 27
Scalable Supervision for Software Agents via Patch Reasoning
The paper introduces R4P, a reasoning‑based supervision method for software agents that eliminates the need for test execution by using a group‑wise training objective to verify multiple patches simultaneously. R4P achieves 72.2% accuracy on the SWE‑bench patch verification task, matching proprietary models, and enables the creation of an execution‑free scaffold called Mini‑SE. Mini‑SE, trained purely with reinforcement learning via R4P, improves Pass@1 from 26.2% to 32.8% over the baseline Qwen3‑32B, demonstrating R4P’s practical utility and scalable performance.
arXiv Machine Learning
Jun 19
Probe-and-Refine Tuning of Repository Guidance for Coding Agents
arXiv:2606. 20512v1 Announce Type: cross Abstract: LLM-based coding agents need higher-level operational knowledge about a repository (which files house which subsystems, how to run the test suite, which workflows have historically led to wrong fixes) that does not exist in the code itself.