A machine learning approach using conditional normalizing flow to address extreme class imbalance problems in...

Area of Science:

• Biomedical informatics
• Machine learning
• Deep learning

Background:

• Supervised machine learning models are used for disease prediction using personal health records.
• Class imbalance in data hinders model training and accuracy.
• Conditional normalizing flow, a deep learning anomaly detection model, is explored to address this challenge.

Purpose of the Study:

• To evaluate the effectiveness of conditional normalizing flow for predicting chronic diseases from personal health records.
• To address the challenge of extreme class imbalance in biomedical datasets.
• To introduce normalizing flow algorithms for tabular biomedical data.

Main Methods:

• Collected personal health records from 706 South Korean citizens, including genetic, medical check-up, and lifestyle data.
• Labeled six chronic diseases: obesity, diabetes, hypertriglyceridemia, dyslipidemia, liver dysfunction, and hypertension.
• Evaluated supervised and semi-supervised models, including conditional normalizing flow, for diabetes classification (2% prevalence) using AUROC and AUPRC, and tested performance with undersampled data for other diseases.

Main Results:

• Conditional normalizing flow achieved higher performance (AUPRC 0.34, AUROC 0.83) than the best supervised model, LightGBM (AUPRC 0.16, AUROC 0.82), for diabetes prediction with a 2% base rate.
• The model outperformed supervised methods for other chronic diseases, even with limited positive samples (e.g., obesity: CNF AUPRC 0.30/AUROC 0.74 vs. LightGBM AUPRC 0.20/AUROC 0.75).
• Performance remained robust even when positive samples were undersampled to a 2% base rate.

Conclusions:

• Conditional normalizing flow is effective for chronic disease prediction using personal health records, especially with limited data.
• This deep learning approach provides a viable solution for sparse data and extreme class imbalances in biomedical contexts.
• The study highlights the potential of normalizing flows for tabular biomedical data analysis.