Socio-Technical Risk-Informed AI-Driven Automation Trustworthiness Evaluation (ST-RATE): Part 1-Conceptualization

The growing integration of artificial intelligence (AI)-driven technologies into nuclear power plant (NPP) operations and maintenance holds significant potential to enhance safety, operational efficiency, and reliability. However, deploying such AI-driven automation technologies in high-consequence settings requires a rigorous, traceable, and risk-informed evaluation of their trustworthiness. Although researchers across various domains have proposed definitions and metrics for AI-driven automation trustworthiness, no agreed-upon formal or quantitative method exists that is suitable for the highly regulated nuclear sector. To address this gap, this paper, the first in a two-part series, presents three contributions: (I) a cross-disciplinary literature review that categorizes existing definitions and evaluation methodologies for AI-driven automation trustworthiness and identifies their inherent limitations with respect to applicability in the nuclear domain; (II) develops a new definition of AI-driven automation trustworthiness as "the degree of confidence that the AI-driven automation system will function as expected across its conditions of use" to address the limitations identified in Contribution I; and (III) building on the new definition, conceptualizes the Socio-Technical Risk-informed AI-driven Automation Trustworthiness Evaluation (ST-RATE) methodology, grounded in three foundational pillars: (a) risk-informed performance expectations, (b) socio-technical context across operational conditions, and (c) uncertainty-based trustworthiness evaluation. While developed primarily for the nuclear domain, the definition and ST-RATE methodology has the potential to be adapted for other safety-critical domains with appropriate modifications. The ST-RATE methodological framework is operationalized and demonstrated through an AI-driven automated firewatch case study in a companion paper (Companion paper Part 2, 2025). ST-RATE is the first-of-its-kind methodology that enables risk-informed and uncertainty-based quantification of trustworthiness despite limited empirical data and without over-reliance on subjective expert judgment, addressing the key limitations of existing approaches identified in the literature review.