arXiv AI

When Messages Go Wrong: Understanding Communication Faults in LLM-Based Multi-Agent Systems

arXiv AI
Aug 18

When Agentic Executions Fail: Detecting and Localizing Runtime Faults from Telemetry

arXiv:2608. 14680v1 Announce Type: new Abstract: Reliability in LLM-based agentic systems is a property of the whole execution (its tool calls, model calls, guardrails, and inter-agent messages), not of the final answer alone, yet evaluating only task outcomes reveals little about how or why a run fails.

By Chenkai Zhang, Yiran Li, Yifang Tian, Michalis Bachras, Hans-Arno Jacobsen
Hugging Face Trending Papers
Aug 27

FaulT-Bench: Towards Benchmarking Network Troubleshooting LLM Agents under Unreliable User Tickets

FaulT-Bench is a new benchmark comprising 200 network troubleshooting scenarios across eight topologies, designed to test large‑language‑model agents on realistic, noisy tickets that may contain false premises or incorrect fault claims. The benchmark includes 72 rewritten tickets that vary reporter confidence and detail, and evaluates agents via an automated harness that scores diagnoses on outcome, fix, and reasoning quality. Results show that while agents perform well on accurate tickets, they degrade sharply on healthy networks with misleading reports, highlighting the importance of ticket wording over content.

arXiv AI
Oct 7

AegisFlow: A Multi-Agent Agentic AI Framework for Autonomous Remediation and Self-Healing in Fragile Data Ecosystems

AegisFlow is a multi‑agent AI framework that automates the detection and remediation of failures in data pipelines, using a Watchdog agent for telemetry and a Repair agent that generates, tests, and deploys code patches via large language models. It employs a non‑intrusive Parallel Shadow Patching approach based on the MAPE‑K loop to validate patches in digital twin environments. Experiments across five failure scenarios show a 98.1% reduction in mean time to repair—from 170 minutes to 3.2 minutes—and a 92% patch success rate, freeing up 98% of data engineering on‑call time for innovation.

By Muhammad Bilal Awan, Zubair Hussain, Abdul Shahid