Using interpretability approaches to update "black-box" clinical prediction models: an external validation study in...

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Prediction Models

Background:

• Machine learning models for clinical prediction face challenges in real-world deployment.
• Validation studies are crucial but often lacking for these models.
• Acute kidney injury (AKI) prediction in cardiac surgery is a critical area for AI application.

Purpose of the Study:

• To validate a machine learning model for AKI prediction in cardiac surgery patients on an external cohort.
• To investigate performance differences between the initial development dataset and the external validation cohort.
• To explore the utility of interpretability methods in understanding and improving model validation.

Main Methods:

• Development of an AKI prediction model using the MIMIC-III dataset.
• External validation of the model on a separate cohort from an American research hospital.
• Application of feature importance-based interpretability methods (global and local) to analyze model behavior.

Main Results:

• Observed performance differences when the model was applied to the external cohort.
• Interpretability methods provided insights into the reasons for unexpected model behavior.
• Feature importance analysis helped experts scrutinize model performance at both global and local levels.

Conclusions:

• Interpretability methods are valuable tools for explaining performance discrepancies in model validation.
• Insights from interpretability can guide model updates for improved generalizability and simplicity.
• Practitioners should consider interpretability methods to enhance the validation process and inform model refinement.