arXiv:2607. 13396v1 Announce Type: new Abstract: What happens to an LLM agent's tool choice when the reliable tool silently changes within an ongoing session?
By Ziwei Ye
SWE-Shift is a new benchmark that tests coding agents in environments where tasks and workspaces can change mid-execution. It presents 120 scenarios across 12 change families, each evaluated at nine intervention points, comparing a Clean continuation (no change) with a Shifted continuation (task-relevant update). The benchmark shows that all evaluated agents perform better on Clean than on Shifted, with significant drops in success rates and zero paired success for certain change types.
By Victor Shea-Jay Huang, Rongqing Li, Zhangyue Yin, Kelong Mao, Yaqi Zhao, Xinyuan Wang, Yao Yin, Lirui Luo, Renrui Zhang, Hongsheng Li
arXiv:2608. 05013v1 Announce Type: cross Abstract: LLM agents are increasingly applied to open-ended everyday requests that span work, study, and life.
By Jingsheng Zheng, Xinyuan Fang, Jintian Zhang, Zhengke Gui, Huajun Chen, Ningyu Zhang
arXiv:2610.16333v1 Announce Type: new
Abstract: As AI agents increasingly act through external tools, tool results are no longer simple outputs but execution artifacts that must remain correctly asso...
By Yutong Zhi, Jie Zhang, Xiuqi Lu, Jiayuan Xu, Jing Shao
The paper introduces RiseHarness, a framework that iteratively refines agent harnesses while keeping model weights fixed. It uses successful and failed runs under the same configuration to analyze failures, combining agent trajectories with runtime monitor traces to understand interactions between model actions and execution components. The framework then searches other harness frameworks for relevant implementations, adapts them, and validates changes through behavioral checks and repeated task evaluation, ultimately integrating mechanisms from multiple frameworks to improve reliability across tasks.
By Ruhan Wang, Yucheng Shi, Zongxia Li, Junyao Yang, Zhongzhi Li, Yue Yu, Haitao Mi, Dongruo Zhou, Leoweiliang
Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and operate over extended horizons. Reported benchmark gains often obscure recurring failure modes documented across otherwise unrelated evaluation efforts.