Multi-Backbone U-Net segmentation and interpretable hybrid deep learning classification for lung cancer detection in...

Lung cancer remains the leading cause of cancer-related mortality worldwide, and its early and accurate diagnosis from chest CT imaging is complicated by overlapping radiological features between benign and malignant lesions, as well as by the limited cross-institution validation and interpretability reported in much of the existing computer-aided diagnosis literature. This study evaluates the effectiveness of U-Net architectures integrated with different pre-trained convolutional neural network (CNN) backbones for automated lung segmentation and lung cancer classification in chest CT images, with an emphasis on rigorous, leakage-free validation and cross-institution generalizability relevant to clinical practice.

A balanced internal dataset of 832 chest CT images (416 cancerous and 416 non-cancerous cases, NIDCH, Bangladesh) was preprocessed using Contrast Limited Adaptive Histogram Equalization (CLAHE) and resized to 224 × 224 pixels. U-Net models with three CNN backbones (VGG16, ResNet50, and Xception) were implemented for lung segmentation, with pseudo-labels verified against expert-annotated masks. A standalone CNN and three hybrid classifiers combining CNN feature extraction with machine learning algorithms (Support Vector Machine, Random Forest, and Gradient Boosting) were then evaluated, alongside a Vision Transformer baseline. All models were assessed using subject-level fivefold cross-validation, followed by fully independent external testing on a second, structurally distinct dataset (IQ-OTH/NCCD; 110 subjects) without retraining. Performance was measured using Dice coefficient, IoU, accuracy, precision, recall, F1-score, and AUC, with 95% confidence intervals and statistical significance testing.

For segmentation, U-Net with the Xception backbone achieved the best overall performance (Dice: 0.9581 ± 0.0104; IoU: 0.9163 ± 0.0167). For classification, the CNN + SVM hybrid model achieved the highest internal cross-validation accuracy (98.21% ± 0.68%) and AUC (0.9881 ± 0.0041), significantly outperforming the standalone CNN (p = 0.041), and retained the strongest performance under external validation (92.73% accuracy), with the smallest accuracy drop among all evaluated models. Ablation analyses confirmed substantial contributions from CLAHE preprocessing and U-Net segmentation, and Grad-CAM analysis showed that segmentation constrains classifier attention to anatomically plausible lung regions.

Integrating U-Net segmentation with hybrid CNN-based classification provides an accurate and reasonably generalizable framework for lung cancer detection, offering a methodological foundation for future clinically oriented computer-aided diagnosis research, pending prospective, multi-institution validation.