Continuous assurance for AI-driven clinical decision support systems

Area of Science:

• Healthcare technology
• Clinical informatics
• Artificial intelligence in medicine

Background:

• Healthcare systems are integrating adaptive and generative AI into clinical processes, transforming healthcare delivery with advanced analytics and decision support.
• Traditional audit and assurance frameworks, designed for static software, are insufficient for AI-CDSS due to systemic, ethical, operational, and governance risks.
• Existing audit models struggle to address adaptive algorithms in complex clinical environments and capture interactions between AI, workflows, and organizational factors.

Purpose of the Study:

• To review the evolving landscape of AI-CDSS audit and its transition from technical validation to socio-technical resilience.
• To identify key paradigm shifts reforming AI-CDSS audit methodologies.
• To propose a new framework for ensuring the safety and trustworthiness of clinical AI systems.

Main Methods:

• Literature review of AI-CDSS audit approaches and governance frameworks.
• Analysis of the limitations of traditional software validation models for AI.
• Development of the Socio-Technical Resilience Assurance for Intelligent Clinical Systems (STRAICS) framework.

Main Results:

• AI-CDSS audit is shifting from model-centric evaluation to ecosystem-level assurance.
• The focus is moving from post-hoc explainability to reasoning traceability and clinical sense-making.
• Audit philosophy is transitioning from static compliance to resilience-oriented monitoring, emphasizing adaptive capacity and safe evolution.
• The proposed STRAICS framework integrates technical robustness, human-machine interaction safeguards, adaptive governance, and transparency-by-design.

Conclusions:

• Traditional audit methods are inadequate for the complexities of AI-CDSS.
• A socio-technical resilience approach is crucial for ensuring patient safety and regulatory compliance.
• The STRAICS framework offers a comprehensive strategy for building trustworthy and effective clinical AI ecosystems.