Global Sensitivity Analysis for Robust XAI: Quantifying Clinical Risk and Prediction Instability in Dermoscopic Image...

Area of Science:

• Dermatology
• Artificial Intelligence
• Medical Imaging

Background:

• Deep learning models achieve high accuracy in predicting malignant skin lesions.
• Operational uncertainties in image acquisition (lighting, device settings, skin characteristics) compromise model reliability in clinical settings.
• This instability creates an unquantified diagnostic risk, hindering safe clinical implementation of AI diagnostic systems.

Purpose of the Study:

• To rigorously quantify the robustness of a convolutional neural network (CNN) architecture against variations in critical optical image parameters.
• To move beyond simple accuracy metrics and provide a robust, risk-quantified assessment of AI model performance.
• To establish the confidence level required for the accreditation and safe clinical deployment of AI-based diagnostic systems.

Main Methods:

• Utilized global sensitivity analysis to assess the CNN architecture's robustness.
• Quantified the impact of five critical optical image parameters on model predictions.
• Focused on identifying and measuring model instability under varying operational conditions.

Main Results:

• The study successfully quantified the instability of the CNN architecture concerning optical image parameters.
• Sensitivity analysis provided a risk-quantified assessment of the model's performance, highlighting areas of vulnerability.
• The findings demonstrate a method for evaluating AI diagnostic system robustness beyond nominal accuracy.

Conclusions:

• Quantifying model robustness is essential for the safe clinical implementation of AI in dermatology.
• Global sensitivity analysis offers a rigorous approach to assessing diagnostic risk associated with AI systems.
• This work provides a foundation for establishing confidence and accreditation for AI-based diagnostic tools in healthcare.