Advancing lung cancer diagnosis: Combining 3D auto-encoders and attention mechanisms for CT scan analysis

ObjectiveThe goal of this study is to assess the effectiveness of a hybrid deep learning model that combines 3D Auto-encoders with attention mechanisms to detect lung cancer early from CT scan images. The study aims to improve diagnostic accuracy, sensitivity, and specificity by focusing on key features in the scans.Materials and methodsA hybrid model was developed that combines feature extraction using 3D Auto-encoder networks with attention mechanisms. First, the 3D Auto-encoder model was tested without attention, using feature selection techniques such as RFE, LASSO, and ANOVA. This was followed by evaluation using several classifiers: SVM, RF, GBM, MLP, LightGBM, XGBoost, Stacking, and Voting. The model's performance was evaluated based on accuracy, sensitivity, F1-Score, and AUC-ROC. After that, attention mechanisms were added to help the model focus on important areas in the CT scans, and the performance was re-assessed.ResultsThe 3D Auto-encoder model without attention achieved an accuracy of 93% and sensitivity of 89%. When attention mechanisms were added, the performance improved across all metrics. For example, the accuracy of SVM increased to 94%, sensitivity to 91%, and AUC-ROC to 0.96. Random Forest (RF) also showed improvements, with accuracy rising to 94% and AUC-ROC to 0.93. The final model with attention improved the overall accuracy to 93.4%, sensitivity to 90.2%, and AUC-ROC to 94.1%. These results highlight the important role of attention in identifying the most relevant features for accurate classification.ConclusionsThe proposed hybrid deep learning model, especially with the addition of attention mechanisms, significantly improves the early detection of lung cancer. By focusing on key features in the CT scans, the attention mechanism helps reduce false negatives and boosts overall diagnostic accuracy. This approach has great potential for use in clinical applications, particularly in the early-stage detection of lung cancer.