The paper studies when to intervene in large language model (LLM) agents that interact with web services. Using the ReInBench dataset of 1,259 agent traces and 1,403 error events, the authors find that 725 errors self‑recover, often within a few steps, and that self‑recovery can aid subsequent error handling and overall task completion. They propose ReCoF, a predictor that decides whether an error will self‑recover, achieving a 9.02% relative improvement in Macro‑F1 over the best baseline.
By Chiming Duan, Tong Jia, Yingxi Chen, Minghua He, Lingzhe Zhang, Ying Li
arXiv:2610.27954v1 Announce Type: new
Abstract: Recent advances in large (vision) language models have enabled computer-use agents that automate digital workflows via fine-grained keyboard and mouse...
By Byeonghun Bang, Sangwoo Park, Jinheon Baek, Kangsan Kim, Youngwan Lee, Sung Ju Hwang
arXiv:2607. 11098v1 Announce Type: cross Abstract: Tool-using LLM agents are mostly evaluated assuming all tools work.
By Aritra Mazumder, Nusrat jahan Lia
The paper re‑evaluates memory‑based self‑improving agents by adding multiple runs to measure variance and by randomizing task order. It finds that agent performance is noisy in complex, multi‑step environments and that improvement depends heavily on the sequence of tasks, revealing a hidden curriculum effect. The authors suggest that underspecification of tasks and environments contributes to this fragility and demonstrate that adding detailed rubrics and feedback can partially mitigate performance drops, though gaps remain.
By Qinyuan Ye, Yu Li, Yada Pruksachatkun, Jiaxin Zhang, Chien-Sheng Wu
arXiv:2607. 10608v1 Announce Type: new Abstract: Memory is becoming a core component of long-horizon AI agents, allowing agents to reuse past experience when operating web browsers, software tools, and other interactive environments.
By Yixiong Chen, Xinyi Bai, Alan Yuille
arXiv:2609.35814v1 Announce Type: cross
Abstract: As browser-use agents improve, benchmarks keep pace by collecting new tasks, websites, and applications, often making tasks longer or more novel. Thi...
By Xunjian Yin, Tianchen Guan, Jinao Wang, Weili Cao, Daisy Xinlei Lin, Royce Cheng-Yue, Keagan Long, Kyle Wong, Bhuwan Dhingra, Xiangjun Wang, Shuyan Zhou
The paper re‑evaluates memory‑based self‑improving agents by running multiple trials and shuffling task orders, revealing that agent performance is noisy and highly sensitive to task sequencing. It shows that implicit curricula in default task orders act as hidden prerequisites for success, and that underspecification of tasks and environments contributes to fragility. Adding detailed rubrics and environment feedback partially mitigates performance drops but significant gaps remain, underscoring the need for stricter evaluation protocols and better human oversight.
arXiv:2608.25920v2 Announce Type: replace
Abstract: As large language model (LLM)-based multi-agent systems (MASs) are increasingly applied to long-horizon complex tasks, their reliability has emerge...
By Zhongwen Luan, Xiaoyu Zhang, Ming Hu, Yue Yang, Jiongchi Yu, Xiaohong Chen
Self-correction is particularly useful when a failure constrains the next repair. Coding agents benefit from this property because compilers, tests, and execution traces turn many failures into typed recovery signals, but broad language-agent tasks often expose only a coarse task failure.
arXiv:2608. 14380v1 Announce Type: new Abstract: Many real-world tasks require LLM agents to interact with their environments over long execution horizons.
By Yu Zhuang, Kefei Chen, Yitong Duan, Shuxin Zheng, Jian Li, Xu-Yao Zhang
arXiv:2609.21423v1 Announce Type: new
Abstract: Online agent deployments produce abundant execution traces, while task-specific verification and expert annotation are costly to scale. We study how to...
By Siyuan Liu (Fudan University, Meituan Longcat Team), Fan Yu (Fudan University, Meituan Longcat Team), Dongyu Ru (Meituan Longcat Team), Yizhu Liu (Meituan Longcat Team), Yifan Yang (Meituan Longcat Team), Xuezhi Cao (Meituan Longcat Team), Xunliang Cai (Meituan Longcat Team), Yixin Cao (Fudan University)
The paper introduces the evidence-action gap, where coding agents persist in tasks even when completion is impossible, despite having sufficient evidence that outside help is needed. To quantify this, the authors present ImpasseBench, a benchmark that injects realistic blockers into repair tasks and marks the step at which evidence becomes sufficient. Across eight models and 19,371 trajectories, they find that many agents still exhaust their budget or make unsafe requests after the evidence threshold, but an escalation tool and external monitor can significantly reduce these failures.
By Tianyu Chen, Xinran Zhao, Vijay Viswanathan, Shoubin Yu, Mingyuan Zhou, Tongshuang Wu