AI Bias and Confounding Risk in Health Feature Engineering for Machine Learning Classification Task

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Clinical Prediction Models

Background:

• Machine learning offers opportunities in healthcare but faces challenges like data variability and AI bias.
• Confounding risks can impact the performance of AI models in clinical settings.
• Real-world health data often has limitations in scope, population coverage, and granularity.

Purpose of the Study:

• To investigate the impact of hidden confounding factors on machine learning model performance in cardiovascular readmission prediction.
• To evaluate the effectiveness of propensity score adjustment in mitigating confounding risks.
• To identify potential confounding factors in predicting patient readmissions.

Main Methods:

• Utilized real-life electronic health record data from the DREAM dataset.
• Applied five machine learning models: k-nearest neighbors (KNN), random forest (RF), decision tree (DT), Catboost, and Xgboost.
• Assessed model performance using Area Under the ROC Curve (AUC) and F1 score, comparing results before and after propensity score adjustment.

Main Results:

• Propensity score adjustment revealed significant performance fluctuations, particularly for patients aged 20-40.
• High-risk pregnant females were identified as a potential confounding factor, with a significantly higher pregnancy rate in the non-readmitted group (χ² = 10.2, p < 0.001).
• Pregnancy status required data from an external system, posing integration challenges.

Conclusions:

• Traditional machine learning pipelines without careful consideration of confounding risks may yield suboptimal clinical classifiers.
• Incorporating propensity score matching is a viable strategy to randomize and account for invisible confounding factors.
• Addressing hidden confounders like pregnancy status is crucial for developing robust and reliable AI tools in healthcare.