arXiv:2609.05774v1 Announce Type: new Abstract: Recent multi-agent LLM systems increasingly rely on graph-structured communication to coordinate specialized agents. We revisit multi-agent orchestrati...
arXiv:2607. 19297v1 Announce Type: new Abstract: This paper is a practitioner guide to graph-based workflow pathways for long-running, stateful, multi-step generative AI systems in business processes.
arXiv:2606. 23797v1 Announce Type: cross Abstract: Graph and multi-agent orchestration frameworks make production large language model (LLM) workflows practical, but they do not by themselves solve conversational continuity when users maintain several interdependent objectives.
The paper introduces RUPA, a trajectory‑level uncertainty quantification framework for large language model agents. RUPA models an agent’s execution as a directed graph of reasoning states, tool interactions, and environment feedback, then propagates uncertainty across this graph to capture long‑range dependencies. Experiments on benchmarks such as τ‑2, Terminal‑Bench‑2, and GAIA show that RUPA outperforms existing methods, enabling earlier failure detection and more reliable agent execution.
Graphectory Viewer is a web-based tool that enables interactive, process‑centric analysis of software‑agent trajectories. It converts heterogeneous raw trajectories into phase‑aware graphs, linking low‑level execution details with higher‑level behavioral structures. The tool supports multiple agent frameworks, offers node‑level inspection, search and filtering over large collections, and Sankey‑style summaries of problem‑solving phase transitions, allowing researchers to inspect individual runs, identify patterns, compare successes and failures, and analyze large corpora beyond final outcomes.
arXiv:2608. 06329v1 Announce Type: cross Abstract: Task-oriented conversational agents are evaluated using curated or automatically generated benchmarks, yet benchmark quality is rarely assessed.
arXiv:2608. 02650v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly rely on external tools to complete complex real-world tasks.
arXiv:2608.21156v1 Announce Type: cross Abstract: LLMs have evolved from language generators to autonomous agents capable of complex, long-horizon tasks. This evolution has produced paradigms includi...
GraphACM introduces a graph-based framework to measure fine-grained action consistency in large language model agents. By merging equivalent states into a directed, action-labeled graph across multiple task rollouts, it enables comparisons of actions at shared decision points and defines node-level and global consistency metrics. Experiments on ALFWorld, AppWorld, and WebShop show that the global consistency score predicts task success with an AUROC of 0.811, outperforming prior consistency baselines and verbalized confidence.
The paper surveys how graphs are increasingly used in large language model systems to support reasoning, knowledge grounding, retrieval, memory, execution, and multi‑agent coordination. It introduces an operation‑level taxonomy that distinguishes the function of each graph operation, the component that computes it, and how its output is subsequently used. The authors compare graph observations with external computations, analyze memory updates, and examine execution dependencies, then propose an evaluation framework that organizes benchmarks, diagnostics, and system costs, concluding with open challenges in adaptive computation, memory maintenance, and dependency‑aware recovery.
DynSTEER is a dynamic stage‑wise trajectory evaluation framework designed for large language model agents performing long‑horizon tasks. It segments rollouts into stages anchored by key actions, uses a path‑tolerant milestone graph to accommodate diverse valid strategies, and adaptively routes queries to multi‑tier judges while halting unrecoverable executions early. Experiments show it improves evaluation discriminability by 85.2% over native methods, separates all model pairs with statistical significance, and saves 34.51% of execution steps on failed rollouts.
As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical. However, current evaluation pipelines remain highly fragmented and tightly coupled, hindering reproducibility and causing redundant engineering.