arXiv AI By Yidian Chen, Yingzi Gu, Natan Vidra, Spurthi Setty, Sharon Zheng

OrchestraBench: Evaluating Multi-Agent Orchestration Failure Modes, Recovery, and Decomposition Quality

Read the original on arXiv AI →

arXiv:2608. 05263v1 Announce Type: new Abstract: Multi-agent orchestration frameworks are moving from demos to production, yet benchmarks typically report task accuracy without diagnosing why a pipeline failed, where a cascade began, or which routing decision caused the breakdown.

Summary generated by The Flow from the publisher's feed. The full article lives at arXiv AI.

arXiv AI
6d ago

CTBench: Evaluating Troubleshooting Capabilities of AI Agents in Realistic Telecom Network Operations

arXiv:2608. 12002v1 Announce Type: new Abstract: Agents are increasingly considered for automating network operations and maintenance, where engineers must diagnose network faults, optimize configurations to enhance services, and reduce operational costs while acting under strict constraints.

By Xingyu Yan, Tingting Dai, Antonio De Domenico, Mohamed Sana, Nicola Piovesan, Changchang Li, Bowen Liu, Kun Jiang, Mengjie Zhang, Dingcheng Shan, Jing-Cheng Pang, Chenwei Wu, Sijie Wu, Lianying Chao, Haoran Cai, Jiantao Ye, Xubin Li, Simon Mark Lucas, Xin Chen
arXiv AI
Jun 2

Monitoring Agentic Systems Before They're Reliable

arXiv:2606. 02494v1 Announce Type: cross Abstract: Agentic systems entering production typically operate as partially integrated assemblies where structural defects, not task-level errors, dominate the failure landscape.

By Marisa Ferrara Boston, Glen Hanson, Effi Georgala, JD Hudgens, Heather Frase