arXiv AI

Tracing Agentic Failure from the Flow of Success

arXiv:2607. 12747v1 Announce Type: new Abstract: Failure attribution for LLM-based agentic systems, i.

arXiv AI

Jun 3

StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems

arXiv:2606. 03467v1 Announce Type: new Abstract: LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks.

By Taiyu Zhu, Yifan Wu, Weilin Jin, Ying Li, Gang Huang

llms agents benchmarks safety

arXiv AI

Jun 2

FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search

arXiv:2606. 00765v1 Announce Type: new Abstract: LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication.

By Md Nakhla Rafi, Md Ahasanuzzaman, Dong Jae Kim, Zhijie Wang, Tse-Hsun Chen

llms agents benchmarks

arXiv AI

Aug 7

TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories

arXiv:2608. 06346v1 Announce Type: new Abstract: LLM-based agentic systems have shown remarkable capabilities in complex domains, while suffering from cascading errors and difficulty in debugging.

By Yunjia Qi, Zehua Yin, Xintong Shi, Hao Peng, Songyuanyi Lu, Yixian Liu, Richeng Xuan, Yuhong Liu, Zhichao Hu, Xiaozhi Wang, Lei Hou, Bin Xu, Juanzi Li

llms agents benchmarks

arXiv Machine Learning

Jun 5

When Evidence is Sparse: Weakly Supervised Early Failure Alerting in Dialogs and LLM-Agent Trajectories

arXiv:2606. 05414v1 Announce Type: cross Abstract: Early failure alerting requires deciding, while a dialog or agent trajectory is still unfolding, whether to flag it as likely to fail.

By Avinash Baidya, Xinran Liang, Ruocheng Guo, Xiang Gao, Kamalika Das

llms agents benchmarks

arXiv AI

Aug 11

FailForge: Distilling Procedural Competence from Persistent Failures into Code Agents

arXiv:2608. 08570v1 Announce Type: new Abstract: Rejection sampling fine-tuning (RFT) is widely used to train code agents by generating trajectories on verifiable software engineering tasks, retaining those that pass the tests, and fine-tuning on the successful rollouts.

By Dongyi Lv, Fushun E, Aichen Cai, Liang Huang, Ya Zhang, Qiuyu Ding, Canhui Wu, Zhi Wang, Yuesong Zhang, Jiaqi Wang, Nan Duan

agents fine-tuning

arXiv AI

Jun 2

From Features to Actions: Explainability in Traditional and Agentic AI Systems

arXiv:2602. 06841v4 Announce Type: replace Abstract: Over the last decade, Explainable AI has primarily focused on interpreting individual model predictions, producing post-hoc explanations that relate inputs to outputs under a fixed decision structure.

By Sindhuja Chaduvula, Jessee Ho, Kina Kim, Aravind Narayanan, Ahmed Y. Radwan, Mahshid Alinoori, Muskan Garg, Dhanesh Ramachandram, Shaina Raza

llms agents

arXiv AI

Jul 14

Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?

arXiv:2607. 09996v1 Announce Type: new Abstract: Automated failure attribution uses LLMs to identify where and why agentic systems fail.

By Jiale Liu, Huajun Xi, Shaokun Zhang, Yifan Zeng, Tianwei Yue, Chi Wang, Jian Kang, Qingyun Wu, Huazheng Wang

llms agents benchmarks

arXiv AI

Jul 1

Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents

arXiv:2606. 31270v1 Announce Type: cross Abstract: Computer-use agents, which leverage multimodal large language models (MLLMs) to operate computers and complete tasks, have attracted significant attention for their utility and versatility.

By Xueqiao Sun, Xiaohan Wang, Ludwig Schmidt, Serena Yeung-Levy, Yuhui Zhang

llms agents fine-tuning multimodal benchmarks

arXiv AI

Jul 16

Experience Memory Graph: One-Shot Error Correction for Agents

arXiv:2607. 13884v1 Announce Type: new Abstract: Large Language Model (LLM) agents have shown remarkable capabilities in autonomous decision-making by generating sequential trajectories of states, actions, and observations.

By Wenjun Wang, Yuchen Fang, Fengrui Liu, Zibo Liang, Kai Zheng

llms agents benchmarks

arXiv AI

Jul 10

Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems

arXiv:2607. 07989v1 Announce Type: cross Abstract: Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures.

By Yufei Xia, Anjun Gao, Yueyang Quan, Zhuqing Liu, Minghong Fang

llms agents fine-tuning benchmarks

arXiv AI

Aug 11

TRACE: TRajectory Attribution for Automated Context Engineering

arXiv:2608. 09153v1 Announce Type: new Abstract: Production AI agents fail when their context sources -- system prompts, knowledge bases, tool descriptions, and procedural skills -- contain errors or gaps.

By Yikai Zhao, Pradeep Kumar Misra, Saurabh Pandey

agents fine-tuning benchmarks

arXiv AI

1d ago

LongRCA Bench: Diagnosing Responsible Roles and Root Causes in Long-Horizon Agent Failures

arXiv:2608. 15242v1 Announce Type: new Abstract: When a long-horizon agent execution fails, outcome-level evaluation reveals the unsuccessful result but not where the decisive error entered the trajectory.

By Yunfei Zhang, Boyu Feng, Changhua Pei, Zexin Wang, Zhihuang Peng, Xinlong Liu, Hengyue Jiang, Difeng Ma, Jiayi Zhang, Yongzhou Yao, Yanan Zhao, Fei Sun, Yintong Huo, Zhaoyang Liu, Jingjing Li, Gaogang Xie, Dan Pei

agents benchmarks

Related stories