Evaluating XAI techniques under class imbalance using CPRD data.

Area of Science:

• Healthcare AI
• Machine Learning Interpretability
• Clinical Decision Support

Background:

• eXplainable Artificial Intelligence (XAI) is critical for regulatory compliance and trust in healthcare AI.
• Post hoc XAI techniques (LIME, SHAP, PDPs) are widely used for interpreting machine learning models in healthcare.
• The reliability of XAI techniques, especially concerning class imbalance in medical data, is not fully understood.

Purpose of the Study:

• To design a framework for evaluating the impact of class imbalance on the consistency of XAI explanations.
• To assess how class imbalance affects explanations from LIME, SHAP, and PDPs across different machine learning models.
• To investigate the reliability of XAI techniques in real-world clinical data scenarios with skewed class distributions.

Main Methods:

• Comparative evaluation of LIME, SHAP, and PDPs using UK primary care data (CPRD).
• Training XGBoost, Random Forest, and MLP models to predict lung cancer risk on both imbalanced and balanced datasets.
• Assessing explanation consistency by comparing models trained on imbalanced versus balanced data.

Main Results:

• Class imbalance significantly affects the reliability and consistency of LIME and SHAP explanations.
• Theoretical analysis explains why LIME and SHAP explanations change with varying class distributions.
• Partial Dependence Plots (PDPs) also show noticeable variations for clinically relevant features under class imbalance.

Conclusions:

• Current XAI techniques exhibit vulnerability when applied to imbalanced medical datasets.
• Consistent model explanations are essential for the trustworthy deployment of machine learning in healthcare.
• Addressing class imbalance is crucial for reliable XAI in clinical applications.