ProgramDistill is a benchmark that evaluates coding agents by having them infer and implement features discovered through interaction with fully functional reference web applications. The benchmark is built by factorizing applications into features of varying granularity, each linked to replayable behaviors that can be executed via a gold patch. Using a mine‑craft‑patch pipeline, the authors discovered 1,975 replay‑verified behaviors across 26 applications and automatically constructed 4,063 tasks, enabling evaluation of nine frontier coding agents, including GPT‑6 Astra and Claude Opus 5, on both full‑application and partial‑application reconstruction tasks.
arXiv:2609.22000v1 Announce Type: new Abstract: Computer-use agents (CUAs) have advanced along two separate lines: graphical interaction and software development through code and the command line. Re...
arXiv:2606. 29957v1 Announce Type: cross Abstract: Most coding-agent benchmarks are static: an agent receives a complete task description up front and is judged only by its final code.
arXiv:2606. 07682v1 Announce Type: cross Abstract: AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex environments.
arXiv:2607. 28591v1 Announce Type: cross Abstract: Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation.
arXiv:2606. 30573v1 Announce Type: new Abstract: We introduce SWE-Interact, a new testbed for evaluating coding agents on multi-turn, interactive, user-driven software engineering tasks.
Scaling coding agents requires a continuing supply of executable data for training, benchmarking, and continuous evaluation. Each task must couple a realistic software state with a specification, development tools, and reliable verification.
arXiv:2608.21833v1 Announce Type: new Abstract: Recent large language models (LLMs) can operate as coding agents that build complete games from natural language requests. Game development is especial...
Wuying-Browser-Agent is a unified framework designed to improve long-horizon browser agents by aligning execution, supervision, optimization, and evaluation. It introduces a structured browser harness, reflection and UI-specialized Curriculum SFT (RUIC‑SFT) for recovery and complex UI interactions, and Divergence‑Aware Online GRPO (DAO‑GRPO) for better credit assignment. The framework is evaluated on BrowserBench—a bilingual real‑web benchmark of 350 tasks—and achieves state‑of‑the‑art results on multiple browser‑use benchmarks, while also transferring well to other agentic tasks.
Terminal-Universe is a framework that converts large collections of terminal-based agent trajectories into reusable, executable environments. By replaying recorded file operations and filling missing files with a completion agent, it reconstructs the original workspace and generates new tasks, scaling them in breadth (cross-workspace queries) and depth (multi-round interactions). The resulting 37.3k task-sufficient environments enable significant performance gains when fine‑tuning language models on terminal‑centric benchmarks.
arXiv:2609.22068v1 Announce Type: new Abstract: Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich sourc...
arXiv:2606. 09426v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools.
arXiv:2607. 15854v1 Announce Type: cross Abstract: Coding agents can fix a failing example without preserving the domain rule that made it fail, so later generations can repeat the same plausible mistake.