Predicting Chronic Kidney Disease Using Hybrid Machine Learning Based on Apache Spark

Area of Science:

• Medical Informatics
• Data Science
• Machine Learning

Background:

• Chronic kidney disease (CKD) poses significant health risks, including cardiovascular disease and end-stage renal disease.
• Early detection and treatment are vital for managing CKD and mitigating its severe complications.
• Machine learning (ML) and Big Data platforms offer powerful tools for accurate disease diagnosis.

Purpose of the Study:

• To propose and evaluate hybrid ML techniques for early CKD detection using Big Data platforms.
• To compare the effectiveness of different feature selection methods integrated with ML classifiers for CKD diagnosis.

Main Methods:

• Implemented hybrid ML models on Apache Spark, integrating Relief-F and chi-squared feature selection with six classification algorithms (DT, LR, NB, RF, SVM, GBT Classifier).
• Evaluated model performance using accuracy, precision, recall, and F1-measure through cross-validation and testing.
• Compared results based on full feature sets versus feature sets selected by Relief-F and chi-squared methods.

Main Results:

• Support Vector Machine (SVM), Decision Tree (DT), and Gradient-Boosted Trees (GBT Classifier) achieved 100% accuracy with selected features.
• The Relief-F feature selection method demonstrated superior performance compared to using full features or chi-squared selected features.
• Hybrid models effectively identified CKD, highlighting the synergy between Big Data, ML, and feature selection.

Conclusions:

• Hybrid ML techniques, particularly with Relief-F feature selection on Big Data platforms, are highly effective for accurate and early CKD detection.
• The proposed approach offers a promising strategy for improving CKD diagnosis and patient outcomes.
• Machine learning provides significant assistance for medical scientists in diagnosing diseases like CKD at their outset.