arXiv:2608. 06346v1 Announce Type: new Abstract: LLM-based agentic systems have shown remarkable capabilities in complex domains, while suffering from cascading errors and difficulty in debugging.
LiteTrajEval is a lightweight architecture designed for budget‑bounded trajectory evaluation of large‑language‑model agents. It creates compact domain‑specific rule profiles offline, preprocesses trajectories online to flag heuristic failure signals, and then uses a single rubric‑guided LLM judge to generate structured diagnostic reports. On public Magnetic‑One‑style and τ‑bench‑style datasets, LiteTrajEval improves failure‑localization alignment with human annotations by 20–35 percentage points and reduces cost and evaluation time by 6× and 8× respectively, and has been deployed in an enterprise agentic platform.
arXiv:2606. 03467v1 Announce Type: new Abstract: LLM-based multi-agent systems exhibit remarkable collaborative capabilities in complex multi-step tasks.
DeFA is a dependency-guided framework that attributes failures in large language model agents by constructing an event dependency graph and a failure propagation graph from protocol relations and semantic dependencies. It identifies violating events, traces their sources and effects, and determines the decisive error, responsible agent, and error category. The method supports long trajectories through segmentation and has shown superior accuracy on text, image, and video tasks, while its diagnostic feedback can improve agent performance on subsequent tasks.
arXiv:2606. 00765v1 Announce Type: new Abstract: LLM-based agents increasingly solve complex tasks through long trajectories involving reasoning steps, tool calls, and inter-agent communication.
arXiv:2607. 12747v1 Announce Type: new Abstract: Failure attribution for LLM-based agentic systems, i.
arXiv:2607. 07989v1 Announce Type: cross Abstract: Large language model (LLM) based multi-agent systems enable complex problem solving through coordinated reasoning and action, but their distributed structure also introduces new challenges in diagnosing system-level failures.
arXiv:2608. 09153v1 Announce Type: new Abstract: Production AI agents fail when their context sources -- system prompts, knowledge bases, tool descriptions, and procedural skills -- contain errors or gaps.
As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical. However, current evaluation pipelines remain highly fragmented and tightly coupled, hindering reproducibility and causing redundant engineering.
arXiv:2607. 13705v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical.
arXiv:2610.27777v1 Announce Type: new Abstract: Tool-using LLM agents increasingly answer e-commerce queries through multi-step tool interactions. The expected output of a tool call can shift between...
The paper introduces a new methodology and analytical framework to diagnose errors in the intermediate execution traces of autonomous Deep Search and Deep Research Large Language Model agents. It presents a taxonomy that classifies these errors into System‑Level and Task‑Level categories, and uses systematic examination of step‑by‑step traces to show how they impact overall performance. The authors also provide empirical experiments and design guidelines aimed at improving the robustness, reasoning, and reliability of these agents.