Comparison between Statistical Models and Machine Learning Methods on Classification for Highly Imbalanced Multiclass...

Area of Science:

• Nephrology
• Biostatistics
• Machine Learning

Background:

• Chronic kidney disease (CKD) diagnosis relies on glomerular filtration rate (GFR) staging.
• Health examination data presents challenges due to highly imbalanced class distributions for CKD stages.
• Accurate classification of CKD stages is crucial for timely intervention and patient management.

Purpose of the Study:

• To compare the classification performance of statistical and machine learning models on imbalanced CKD data.
• To identify the most effective model for accurately classifying six distinct CKD stages.
• To evaluate models using comprehensive performance metrics beyond simple accuracy.

Main Methods:

• Utilized Korean National Health Insurance Service cohort data.
• Compared multinomial logistic regression (LR), ordinal LR, random forest (RF), and autoencoder (AE) models.
• Employed 10-fold cross-validation with stratified sampling for imbalanced data.
• Evaluated performance using accuracy, sensitivity, specificity, precision, F1-Measure, macro-average, and micro-weighted values.

Main Results:

• RF and AE demonstrated superior accuracy compared to multinomial and ordinal LR.
• Standard accuracy can be misleading on imbalanced datasets, necessitating a multi-metric evaluation.
• Autoencoder (AE) achieved the best performance across all evaluated classification indices.

Conclusions:

• Autoencoders (AE) are the optimal model for classifying CKD stages, particularly with imbalanced health datasets.
• A comprehensive evaluation using sensitivity, specificity, precision, and F1-Measure is essential for imbalanced data.
• This study provides a robust framework for selecting machine learning models in clinical diagnostics.