arXiv:2607. 16345v1 Announce Type: cross Abstract: Modern agentic systems increasingly rely on skills: installable packages of natural language and code that teach an LLM agent to perform a domain task.
arXiv:2607. 28871v1 Announce Type: cross Abstract: When a repair agent runs a test and sees it pass, the result is treated as evidence about the reported defect.
arXiv:2610.33667v1 Announce Type: new Abstract: Resolving a repository-level issue typically requires a software-engineering (SWE) agent to localize the faulty code, reproduce the failure, and patch...
The Agent Error Dataset (AED) presents 50,228 error–diagnosis pairs collected from 9,961 source tasks across 33 environments, 19 harness families, and 23 policy models in text‑based agent systems. A five‑stage Agentic Error‑to‑Training (AET) pipeline generates diagnoses and proposed corrections, verifies them against recorded evidence, and creates separate training views for diagnosis and actor recovery. Experiments show that first‑proposal corrections improve verifier pass rates from 18.4% to 51.1%, and fine‑tuning with full‑diagnosis data raises Qwen3‑8B’s exact‑step agreement from 47.2% to 63.6% on a holdout set.
SWEPatchTest is a benchmark that evaluates software tests by measuring their ability to reproduce issues, accept correct patches, and reject incorrect ones across 619 tasks and 82,579 candidate patches. It introduces the Joint Validation Rate metric, revealing that even the best configurations only achieve 9.21% success when all three criteria are met. Using these validation scores to select demonstrations for supervised fine‑tuning can roughly triple the Joint Validation Rate and enhance downstream repair performance on multiple SWE‑bench datasets.
arXiv:2607. 08124v1 Announce Type: cross Abstract: The behavior of an LLM agent is determined not only by the underlying model, but also by its harness: the executable program that constructs context, invokes tools, verifies intermediate results, and recovers from failures.
The paper introduces VERSE, a Verified Self‑Evolving optimizer that enhances LLM agent harnesses by allowing the optimizer to test edits, replay failures, and perturb steps while tracking fixes and regressions. VERSE builds its own tools for failure analysis, verification, training audits, and workflow control, and uses this feedback to revise the harness’s prompts, skills, tools, hooks, and notes without changing model weights. In experiments across five executors and multiple languages, VERSE improves all evaluated harness optimizers, achieving higher accuracy on held‑out and out‑of‑distribution tasks compared to the strongest baselines.
arXiv:2608. 08570v1 Announce Type: new Abstract: Rejection sampling fine-tuning (RFT) is widely used to train code agents by generating trajectories on verifiable software engineering tasks, retaining those that pass the tests, and fine-tuning on the successful rollouts.
arXiv:2610.07898v1 Announce Type: new Abstract: Repository-level software engineering (SWE) is a challenging long-horizon setting: agents must reason over extended interactions, use tools, and adapt...
The paper introduces R4P, a reasoning‑based supervision method for software agents that eliminates the need for test execution by using a group‑wise training objective to verify multiple patches simultaneously. R4P achieves 72.2% accuracy on the SWE‑bench patch verification task, matching proprietary models, and enables the creation of an execution‑free scaffold called Mini‑SE. Mini‑SE, trained purely with reinforcement learning via R4P, improves Pass@1 from 26.2% to 32.8% over the baseline Qwen3‑32B, demonstrating R4P’s practical utility and scalable performance.
SWE‑Duel proposes a self‑scaling benchmark for coding agents, where tasks and oracles are generated at evaluation time to prevent data leakage and adapt difficulty to participant performance. In the arena, two agents take turns as Red and Blue: Red proposes a feature, injects a subtle bug, and must repair it blind, while Blue reviews the pull request without knowing the bug and must pass all tests. A tournament of 11 model‑harness participants on 15 repositories across five languages showed GPT‑5.5/Codex CLI as the top solver and Claude Opus 4.8/mini‑swe‑agent as the top task author, with newer model versions producing harder tasks for older ones.
arXiv:2610.18755v1 Announce Type: cross Abstract: Reinforcement learning (RL) from unit tests is the common recipe for post-training code models, but a unit test only asks whether the final program p...