Explaining multivariate molecular diagnostic tests via Shapley values.

Area of Science:

• Biostatistics
• Bioinformatics
• Machine Learning

Background:

• Machine learning (ML) is crucial for extracting insights from molecular data to develop diagnostic tests.
• Algorithmic explainability is essential for understanding how ML models generate classifications.
• Shapley values provide a method for explaining ML model outputs based on input data.

Purpose of the Study:

• To calculate and interpret exact Shapley values for a clinical molecular diagnostic test (VeriStrat®).
• To compare exact Shapley values with approximation methods like LIME and SHAP.
• To assess the utility of Shapley values for sample-level explanations and patient subgroup identification.

Main Methods:

• Exact Shapley values were computed for the VeriStrat® test using data from 256 patients.
• Standard approximation techniques, including LIME and SHAP-based methods, were employed.
• Results from exact Shapley values were compared against those from approximation methods.

Main Results:

• Exact Shapley values demonstrated that attribute importance for classification varied by sample.
• Interpretability at the sample/patient level was achieved using exact Shapley values and interaction metrics.
• Approximation methods (LIME, SHAP) produced quantitatively and qualitatively different results compared to exact Shapley values.

Conclusions:

• Shapley values effectively determine attribute importance in molecular diagnostic tests for individual patients.
• Shapley value profiles can define patient subgroups, potentially guiding translational research.
• Caution is advised when using approximate Shapley value methods due to potential discrepancies with exact values, especially with correlated molecular data and small training sets.