← Back to all news
arXiv AI

Reliability Engineering for AI Systems: Challenges, Methods, and Directions

The paper "Reliability Engineering for AI Systems: Challenges, Methods, and Directions" discusses how AI reliability extends beyond correct outputs to include consistent operation of retrieval, memory, tool use, permissions, human oversight, and inter-system interactions, especially for generative AI. It adapts traditional reliability engineering techniques—such as failure definitions, operational envelopes, FMEA, accelerated testing, field monitoring, and reliability growth—to AI, proposing a four-level diagnostic framework and a TEVV (Test, Evaluation, Verification, Validation) approach. The authors also introduce the SMART framework for statistical guidance and reference NIST’s AI risk management framework, illustrating their concepts with three case studies and emphasizing the urgent need for new measurements and safety guardrails as AI systems evolve rapidly.

Related stories

arXiv AI
Sep 15

AI Deployment Accountability Engineering: A Vision for Accountable AI in Safety-Critical Socio-Technical Systems

The paper proposes AI Deployment Accountability Engineering (ADAE), a new subdiscipline focused on establishing measurable, continuous, and actionable accountability for AI systems once they are deployed. ADAE treats accountability as a deployment-layer property, aiming to ensure systems remain within acceptable risk limits, identify failure contexts, attribute failures across technical and human components, and translate technical failures into downstream consequences. The authors outline a research agenda built around four pillars—structured discovery of context-dependent failure modes, privacy-preserving accountability measurement, system-level risk analysis for agentic AI, and translation of technical failures into operational and institutional risks—to support timely intervention in safety-critical socio-technical environments.

By Murat Kantarcioglu
agentssafety
More like this →
arXiv AI
Oct 26

Trial by Trace: Benchmarking AI Agent Harnesses for Reliability and Failure Attribution

The paper introduces TraceTrial-Bench, a benchmark that evaluates the reliability of AI agent harnesses across seven components—context, execution, governance, lifecycle, observability, tools, and verification—using 95 expert-level tasks with executable or state-based ground truth. It employs a Trajectory Tracing module to record agent execution, a two-layer evaluation framework to separately assess agent performance and harness reliability, and a Multi-source Evidence Acquisition module to gather evidence for LLM-based evaluation. The benchmark’s analysis of 3,420 runs across four harnesses and nine models shows harness scores between 59.8% and 72.0%, with 31.1% of runs succeeding despite failing at least one harness criterion, highlighting the need for auditable and reproducible harness evaluation.

By Zhiyong Cao, Hengjie Zhu, Yongxin Wang, Xuchen Li, Jintao Li, Shuxia Liu, Dunqiang Liu, Bo Wang, Kairan Zhang, Cong Lin, JiangDi Yu, Siyuan Liu, Sijia Yao, Yi-Liang Wei, Lingxu Ma, Aijia Wang, Anxin Chen, Zixian Wei, Xiang Ruan, Ruizhi Xu, Ruqian Shi, Baihua Xiao, Sicheng Zhou
llmsagentsbenchmarks
More like this →
arXiv AI
Jun 6

Explainable AI-Driven Cyber Risk Analytics and Model Reliability Assessment for Intelligent Governance of U.S. Critical Infrastructure: An XGBoost and SHAP-Based Intrusion Detection Framework

arXiv:2606. 05710v1 Announce Type: cross Abstract: The increasing penetrations of the critical infrastructure sector in the United States with intelligent digital technologies have greatly increased exposure to advanced cyber adversaries and operational vulnerabilities.

By B. M. Taslimul Haque, Md. Arifur Rahman, Md. Serajul Kabir Chowdhury Rubel, Md. Iqbal Hossan
safety
More like this →