arXiv AI

AgentTrails: Towards Trust and Reuse for Agentic Tasks

arXiv:2607. 18816v1 Announce Type: cross Abstract: LLM-powered agents increasingly tackle complex tasks by invoking tools, querying databases, executing code, and manipulating intermediate artifacts.

arXiv AI
Jun 16

From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents

arXiv:2606. 04990v2 Announce Type: replace-cross Abstract: Large language model (LLM)-based agents are evolving from passive text generators into autonomous systems capable of planning, tool use, retrieval, memory access, environmental interaction, and multi-agent collaboration.

By Yiqi Wang, Jiaqi Zhang, Taotao Cai, Zirui Liu, Qingqiang Sun, Zequn Sun, Zhangkai Wu, Manqing Dong, Mingkai Zhang, Xuefei Yin, Yanming Zhu
arXiv AI
Aug 20

LEDGER: Claim-to-Evidence Trace Graphs for Auditing LLM Agents

LEDGER is a tracing and review system for large language model agents that constructs layered trace graphs from observed sessions. It groups raw trace records into Evidence Nodes and Workflow Nodes, anchors artifacts as evidence, and adds typed semantic edges linking claims to supporting actions, artifacts, and checks. The resulting traces reveal workflow decisions, artifact lineage, repair steps, validation coverage, and claim‑support paths for evidence‑centered audit.

By Daehong Kim, Haichao Miao, Shusen Liu
arXiv AI
Aug 19

Graphectory Viewer: A Tool for Process-Centric Analysis of Agentic Software Trajectories

Graphectory Viewer is a web-based tool that enables interactive, process‑centric analysis of software‑agent trajectories. It converts heterogeneous raw trajectories into phase‑aware graphs, linking low‑level execution details with higher‑level behavioral structures. The tool supports multiple agent frameworks, offers node‑level inspection, search and filtering over large collections, and Sankey‑style summaries of problem‑solving phase transitions, allowing researchers to inspect individual runs, identify patterns, compare successes and failures, and analyze large corpora beyond final outcomes.

By Charlie Jyu, Shuyang Liu, Reyhaneh Jabbarvand
arXiv AI
6d ago

The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge

The paper introduces KNOWS, a benchmark for evaluating web agents that act as assistants by retrieving, synthesizing, and presenting information across complex, multi-step browser tasks. It outlines a task design rubric, evaluation protocol combining deterministic checks with LLM judgments, and reports that current agents achieve only modest success, with the best performing agent succeeding on less than 3% of tasks. The study highlights significant gaps in agents’ tool use, visual understanding, and long‑horizon reasoning.

By Alexander Gill, Md Farhan Ishmam, Xuyen Nguyen, Neha Bhat, Parker Henry DeYoung, Fateme Hashemi Chaleshtori, Nathan Stringham, Kenneth Marino, Ana Marasovi\'c