Variable importance analysis with interpretable machine learning for fair risk prediction.

Area of Science:

• Clinical informatics
• Machine learning
• Statistical modeling

Background:

• Machine learning (ML) methods are widely used for variable importance assessment.
• However, traditional "black box" ML models often lack stability with small sample sizes and do not formally identify non-important variables.
• This limits their reliability and interpretability in critical applications like clinical risk prediction.

Purpose of the Study:

• To introduce and evaluate the Shapley variable importance cloud (ShapleyVIC) as a novel method for robust and interpretable variable importance assessment.
• To address the limitations of existing ML methods, particularly in scenarios with limited sample sizes and the need for formal significance testing.
• To assess the potential of ShapleyVIC in improving the fairness and accuracy of clinical risk prediction models.

Main Methods:

• ShapleyVIC assesses variable importance using an ensemble of regression models.
• This ensemble approach enhances model robustness and interpretability.
• The method incorporates uncertainty estimation to formally test the significance of variable importance.

Main Results:

• ShapleyVIC successfully identified important variables in a clinical study where Random Forest and XGBoost models failed.
• The method demonstrated robustness by reproducing findings from smaller subsamples, even when statistical power diminished.
• ShapleyVIC correctly identified race as non-significant, supporting its exclusion from the prediction model and contrasting with conventional stepwise methods.

Conclusions:

• ShapleyVIC provides a robust and interpretable solution for variable importance assessment in machine learning.
• Its ability to formally test significance and handle limited data enhances reliability.
• ShapleyVIC holds significant potential for contributing to fairer and more accurate clinical risk prediction.