arXiv:2610.09633v1 Announce Type: new Abstract: The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engine...
The evaluation of coding agents generally strives to be as realistic as possible. In our study, we collect 4,782 agent sessions of real software engineers in JetBrains IDEs, which we call Production S...
arXiv:2602.18998v2 Announce Type: replace Abstract: LLM agents are increasingly expected to operate as general-purpose systems that resolve real-world user requests, yet their dynamic scaling behavio...
TEX is a new test‑time scaling framework that uses execution‑driven cross‑validation to generate and refine tests for language‑model agents. By running agents in parallel and cross‑executing each test against every patch, TEX builds a $K imes K$ feedback matrix that guides objective refinement without sharing verbose traces. On the SWT‑Bench Verified benchmark, TEX reaches 86.4% accuracy—setting a new state‑of‑the‑art and improving over single‑agent baselines by 9.0%—and even random selections from its outputs outperform the previous best score.
arXiv:2608. 00106v1 Announce Type: new Abstract: Agentic systems must decide not only what answer to produce, but which reasoning and execution operations should precede it.
arXiv:2607. 11399v1 Announce Type: cross Abstract: Large language model agents are increasingly executed not by a single model call, but by an execution harness that manages observation, context, control, action, state, and verification.
arXiv:2606. 07682v1 Announce Type: cross Abstract: AI agents are increasingly expected to complete long-horizon workflows that require sustained progress over hours, millions of tokens, and complex environments.
arXiv:2608. 00107v1 Announce Type: new Abstract: Agentic systems must repeatedly decide whether to answer directly, decompose a task, invoke a tool, execute code, delegate to a specialist, verify an intermediate result, or recover from failure.
arXiv:2605.18859v3 Announce Type: replace-cross Abstract: LLM routing matters most in long-horizon applications such as coding agents, deep research systems, and computer-use agents, where a single u...
arXiv:2607. 20468v1 Announce Type: new Abstract: AI agents are increasingly used to automate research and development tasks, yet existing benchmarks typically evaluate them on prescribed workflows or narrow action spaces.
AgentHop is a diagnostic benchmark comprising 1,011 multiple‑choice questions that test agentic multi‑hop question answering over scientific citation graphs. It provides a controlled seven‑tool sandbox with fixed token, turn, and tool‑call limits, allowing researchers to dissect model performance across four operational axes: retrieval, synthesis, tool‑call, and resource management. Analysis of 19 models shows distinct family‑specific behaviors, such as GPT models committing early, Anthropic and GLM checkpoints verifying before committing, DeepSeek and Kimi over‑searching, and Gemini‑3 Pro maintaining balance, while within‑family differences are revealed in retrieval versus synthesis strategies.
arXiv:2607. 22465v1 Announce Type: cross Abstract: Routing to select large language models (LLMs) with different cost-quality trade-offs has become a fundamental deployment feature of enterprise AI.