arXiv AI
Jul 14
Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?
arXiv:2607. 09996v1 Announce Type: new Abstract: Automated failure attribution uses LLMs to identify where and why agentic systems fail.
Adaptive Influence Graphs for Failure Attribution in Multi-Agent Systems
The paper introduces Adaptive Influence Graphs (AIGs), a two‑stage framework that first converts a failed trace into a structured graph and then navigates it to pinpoint the critical error in multi‑agent large language model systems. Experiments across multiple models demonstrate that richer trace representations and adaptive graph construction improve failure attribution, with AIGs achieving state‑of‑the‑art results on the Who&When benchmark. The study shows that both the diagnosing model and the way traces are represented and explored are crucial for accurate failure attribution.
Related stories
arXiv AI
Jun 2
FALAT: Tracing Failures in LLM Agent Trajectories via Dependency-Guided Search
arXiv:2606. 00765v1 Announce Type: new Abstract: LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication.
arXiv AI
Jul 10
Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems
arXiv:2607. 07989v1 Announce Type: cross Abstract: Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures.
arXiv AI
Jul 15
Tracing Agentic Failure from the Flow of Success
arXiv:2607. 12747v1 Announce Type: new Abstract: Failure attribution for LLM-based agentic systems, i.
arXiv AI
Aug 11
TRACE: TRajectory Attribution for Automated Context Engineering
arXiv:2608. 09153v1 Announce Type: new Abstract: Production AI agents fail when their context sources -- system prompts, knowledge bases, tool descriptions, and procedural skills -- contain errors or gaps.
arXiv AI
Aug 7
TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories
arXiv:2608. 06346v1 Announce Type: new Abstract: LLM-based agentic systems have shown remarkable capabilities in complex domains, while suffering from cascading errors and difficulty in debugging.
arXiv AI
Jun 3
StepFinder: A Temporal Semantic Framework for Failure Attribution in Multi-Agent Systems
arXiv:2606. 03467v1 Announce Type: new Abstract: LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks.
arXiv AI
Jun 2
Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories
arXiv:2606. 02060v1 Announce Type: new Abstract: Deep-research agents solve tasks through long trajectories of search, tool use, evidence inspection, and answer synthesis.
arXiv AI
Aug 10
Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework
arXiv:2608. 06909v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly operate through long-horizon trajectories involving user instructions, tool use, external observations, and memory.
arXiv AI
Jun 2
POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems
arXiv:2606. 02282v1 Announce Type: new Abstract: Orchestrating Large Language Models into Multi-Agent Systems (LLM-MAS) has unlocked remarkable reasoning capabilities, yet emergent failures and hallucinations that resist characterisation block their deployment in safety-critical domains -- a gap made legally untenable by emerging AI regulation.
arXiv AI
Jun 9
REFLECT: Intervention-Supported Error Attribution for Silent Failures in LLM Agent Traces
arXiv:2606. 09071v1 Announce Type: new Abstract: Large language model (LLM) agents now solve complex tasks through long plan-and-execution traces, yet the ability to locate errors in a completed traces still lags far behind, especially in the \emph{silent failure} regime.
arXiv AI
Jun 2
From Features to Actions: Explainability in Traditional and Agentic AI Systems
arXiv:2602. 06841v4 Announce Type: replace Abstract: Over the last decade, Explainable AI has primarily focused on interpreting individual model predictions, producing post-hoc explanations that relate inputs to outputs under a fixed decision structure.