Lung cancer risk prediction using augmented machine learning pipelines with explainable AI.

Area of Science:

• Oncology
• Medical Informatics
• Computer Science

Background:

• Lung cancer is a leading cause of cancer mortality globally, necessitating improved diagnostic accuracy.
• Machine learning (ML) shows potential for lung cancer prediction, but clinical datasets often suffer from class imbalance, hindering classifier performance.
• Class imbalance leads to biased predictions and reduced accuracy in ML models for medical diagnosis.

Purpose of the Study:

• To evaluate the effectiveness of various data augmentation techniques in addressing class imbalance for lung cancer prediction using ML.
• To compare the performance of different augmentation-classifier combinations on a small, imbalanced lung cancer dataset.
• To enhance the accuracy and reliability of ML-based lung cancer risk prediction models.

Main Methods:

• Applied several data augmentation techniques to a small lung cancer dataset characterized by significant class imbalance.
• Trained and evaluated various ML classifiers in combination with these augmentation methods.
• Utilized K-Means SMOTE combined with a Multi-Layer Perceptron (MLP) classifier for comparative analysis.
• Employed LIME (Local Interpretable Model-agnostic Explanations) for model interpretability.

Main Results:

• The combination of K-Means SMOTE data augmentation and a Multi-Layer Perceptron classifier yielded the highest performance.
• Achieved a classification accuracy of 93.55% and an Area Under the Receiver Operating Characteristic Curve (AUC-ROC) score of 96.76%.
• This augmentation-classifier pair significantly outperformed other combinations tested, demonstrating the impact of optimized data augmentation.

Conclusions:

• Data augmentation techniques are crucial for mitigating class imbalance issues in ML-based medical prediction tasks.
• The K-Means SMOTE and MLP combination represents a promising approach for improving lung cancer risk prediction accuracy.
• Further validation with larger, more representative datasets is warranted to translate these findings into clinical tools.