Predicting disease risks from highly imbalanced data using random forest.

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Predictive Analytics

Background:

• Healthcare data, such as the Healthcare Cost and Utilization Project (HCUP) dataset, contains valuable information for predicting individual disease risk.
• Accurate disease risk prediction can enhance healthcare management, personalized communication, and clinical decision support systems.

Purpose of the Study:

• To develop and evaluate a machine learning methodology for predicting disease risk using historical medical diagnosis data.
• To assess the performance of various classification algorithms in identifying individuals at risk for chronic diseases.

Main Methods:

• Utilized the National Inpatient Sample (NIS) data from HCUP for training predictive models.
• Employed an ensemble learning approach with repeated random sub-sampling to manage highly imbalanced HCUP data.
• Compared Random Forest (RF) classifiers against Support Vector Machine (SVM), bagging, and boosting for predicting eight chronic diseases.

Main Results:

• The Random Forest (RF) ensemble learning method demonstrated superior performance compared to SVM, bagging, and boosting.
• RF achieved a higher Area Under the Receiver Operating Characteristic curve (AUC) for predicting eight disease categories.
• RF offers the advantage of calculating variable importance, providing insights into predictive factors.

Conclusions:

• Combining repeated random sub-sampling with RF effectively addresses class imbalance issues in healthcare datasets.
• The proposed method achieved promising results, predicting eight disease categories with an average AUC of 88.79% using national HCUP data.