The paper studies when to intervene in large language model (LLM) agents that interact with web services. Using the ReInBench dataset of 1,259 agent traces and 1,403 error events, the authors find that 725 errors self‑recover, often within a few steps, and that self‑recovery can aid subsequent error handling and overall task completion. They propose ReCoF, a predictor that decides whether an error will self‑recover, achieving a 9.02% relative improvement in Macro‑F1 over the best baseline.
LLM agents interact with Web services to perform tasks on users' behalf. Errors during these interactions can disrupt execution and prevent task completion, motivating external intervention. However,...
arXiv:2610.27954v1 Announce Type: new Abstract: Recent advances in large (vision) language models have enabled computer-use agents that automate digital workflows via fine-grained keyboard and mouse...
arXiv:2607. 11098v1 Announce Type: cross Abstract: Tool-using LLM agents are mostly evaluated assuming all tools work.
arXiv:2607. 10608v1 Announce Type: new Abstract: Memory is becoming a core component of long-horizon AI agents, allowing agents to reuse past experience when operating web browsers, software tools, and other interactive environments.
The paper re‑evaluates memory‑based self‑improving agents by adding multiple runs to measure variance and by randomizing task order. It finds that agent performance is noisy in complex, multi‑step environments and that improvement depends heavily on the sequence of tasks, revealing a hidden curriculum effect. The authors suggest that underspecification of tasks and environments contributes to this fragility and demonstrate that adding detailed rubrics and feedback can partially mitigate performance drops, though gaps remain.
arXiv:2609.35814v1 Announce Type: cross Abstract: As browser-use agents improve, benchmarks keep pace by collecting new tasks, websites, and applications, often making tasks longer or more novel. Thi...
arXiv:2608.25920v2 Announce Type: replace Abstract: As large language model (LLM)-based multi-agent systems (MASs) are increasingly applied to long-horizon complex tasks, their reliability has emerge...
The paper re‑evaluates memory‑based self‑improving agents by running multiple trials and shuffling task orders, revealing that agent performance is noisy and highly sensitive to task sequencing. It shows that implicit curricula in default task orders act as hidden prerequisites for success, and that underspecification of tasks and environments contributes to fragility. Adding detailed rubrics and environment feedback partially mitigates performance drops but significant gaps remain, underscoring the need for stricter evaluation protocols and better human oversight.
The paper introduces the evidence-action gap, where coding agents persist in tasks even when completion is impossible, despite having sufficient evidence that outside help is needed. To quantify this, the authors present ImpasseBench, a benchmark that injects realistic blockers into repair tasks and marks the step at which evidence becomes sufficient. Across eight models and 19,371 trajectories, they find that many agents still exhaust their budget or make unsafe requests after the evidence threshold, but an escalation tool and external monitor can significantly reduce these failures.
arXiv:2610.27645v1 Announce Type: new Abstract: As computer-use agents (CUAs) move to long-horizon workflows spanning multiple applications, undetected errors can silently corrupt persistent state an...
arXiv:2608. 14380v1 Announce Type: new Abstract: Many real-world tasks require LLM agents to interact with their environments over long execution horizons.
arXiv:2606. 01416v1 Announce Type: new Abstract: Tool-augmented large language model (LLM) agents rely on orchestration layers that coordinate planning, retrieval, tool invocation, validation, memory, and recovery.