Fivefold Cross-Validation Approach in Evaluating the Robustness of Machine Learning Models for Prediction of...

Esophageal cancer is a significant health concern worldwide, accounting for 3.1% of all cancer burdens and 5.5% of all cancer-related deaths. Due to its impact, interest in adopting advanced methodologies has increased. Machine learning techniques offer a promising approach for gaining a deeper understanding of this disease.

The study is based on a case-control study design, with a total of 400 case-control subjects equally distributed. The study examined various machine learning-based prediction models, and for each model, several performance metrics, including accuracy, precision, F1 score, recall, and ROC-AUC, were evaluated. To optimize each model and determine the importance of the factors, a fivefold cross-validation technique was employed, and the ranking of feature importance was performed based on the weights in each model.

This study identified the Extra Tree Classifier model as the optimal approach for predicting esophageal cancer, with a model accuracy of 87.50%, a sensitivity of 92.5%, and a specificity of 80%. Compared with the top 10 risk factors on the basis of weight of feature importance, the model yielded an ROC-AUC value of 0.913, representing a substantial improvement of 10.1% over the baseline value of the traditional risk prediction model (ROC-AUC 0.812; 95% CI 0.59-0.94).

The extra tree classifier model exhibited higher predictability and accuracy in identifying significant predictors of esophageal cancer. The incorporation of this machine learning-based model presents exciting opportunities for policymakers to focus on specific risk factors.