Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
Addressing the inadequacy of average benchmark accuracy for quantifying operational dependability, the authors adapt classical industrial reliability engineering methods—including FMEA, accelerated testing, and FRACAS—to autonomous and generative AI systems. This paradigm shift accounts for dynamic failure modes across memory, tool use, and reasoning loops that capability evaluations routinely miss. The resulting framework establishes a four-tier failure taxonomy (component, operational-loop, agentic-conduct, and network/governance) integrated with SMART statistical testing and the NIST AI RMF, validated through empirical case studies in adversarial vision, perception cascades, and autonomous vehicle disengagements.
High benchmark accuracy masks catastrophic runtime drift: borrowing classic industrial reliability engineering—from FMEA to sequential monitoring—is essential to quantify when, where, and how autonomous agentic loops will fail.
AI reliability concerns whether an AI system performs its intended function dependably over a stated period and under stated operating conditions, with stated evidence. As these systems become more autonomous, that function includes more than a correct output. Retrieval, memory, tool use, permissions, human oversight, and interactions among systems must operate consistently and safely, and, for generative systems, so must the reasoning process that produces the output. Average benchmark accuracy measures capability; it does not quantify this broader reliability claim. This paper adapts established reliability engineering methods, from failure definitions and operational envelopes to FMEA, accelerated testing, field monitoring, and reliability growth, to AI systems. A four-level diagnostic framework classifies failures as component, operational-loop, agentic-conduct, or network and governance failures. Test, evaluation, verification, and validation (TEVV), sequential monitoring, and FRACAS create and refresh evidence. SMART provides statistical guidance for measurement, analysis, assessment, and test planning; the NIST AI Risk Management Framework provides organizational guidance for governance, evaluation, monitoring, and mitigation. Three cases illustrate the program: adversarial testing of a convolutional neural network, perception-error propagation, and autonomous-vehicle disengagements. Established reliability engineering provides a usable foundation; new measurements and safety guardrails are still needed as these systems are self-evolving.