A comprehensive data level analysis for cancer diagnosis on imbalanced data.

Area of Science:

• Medical Data Analysis
• Computational Biology
• Oncology

Background:

• Early cancer diagnosis is vital due to its high mortality rate.
• Class imbalance in cancer datasets poses a significant challenge, leading to misclassification and potentially fatal outcomes for patients.
• Existing data analysis methods struggle with imbalanced datasets, necessitating advanced techniques.

Purpose of the Study:

• To comprehensively study the consequences of class imbalance in cancer data.
• To evaluate the effectiveness of various oversampling and undersampling techniques in improving cancer diagnosis.
• To identify optimal balancing techniques and classifiers for different cancer types.

Main Methods:

• Employed 18 balancing algorithms (oversampling and undersampling) on 15 SEER cancer datasets.
• Utilized four classifiers: RIPPER, MLP, KNN, and C4.5.
• Assessed performance using AUC and Friedman statistical tests.

Main Results:

• Balancing techniques significantly improved classifier performance in 90% of cases, as measured by AUC.
• Oversampling techniques generally yielded better results than undersampling techniques.
• Each cancer dataset exhibited unique responses to different balancing techniques and classifiers.

Conclusions:

• Class balancing is essential for improving the accuracy of cancer diagnosis from imbalanced datasets.
• Oversampling methods are more effective than undersampling methods for cancer data.
• Tailoring balancing techniques and classifiers to specific cancer types is recommended for optimal diagnostic performance.