Computer-Aided Diagonosis for Colorectal Cancer using Deep Learning with Visual Explanations

Detection, diagnosis, and removal of colorectal neoplasms are well-accepted colorectal cancer prevention methods. Although promising endoscopic imaging techniques including narrow-band imaging have been developed, these techniques are operator-dependent and interpretations of the results may vary. To overcome these limitations, we applied deep learning to develop a computer-aided diagnostic (CAD) system of colorectal adenoma. We collected and divided 3000 colonoscopic images into 4 categories according to the final pathology, normal, low-grade dysplasia, high-grade dysplasia, and adenocarcinoma. We implemented three convolutional neural networks (CNNs) using Inception-v3, ResNet-50, and DenseNet-161 as baseline models. We further altered the models using several strategies: replacement of the top layer, transfer learning from pre-trained models, fine-tuning of the model weights, rebalancing and augmentation of the training data, and 10-fold cross-validation. We compared the outcomes of the three CNN models to those of two endoscopist groups having different years of experience, and visualized the model predictions using Class Activation Mapping (CAM). The CNN-CAD achieved the best performance in our experiments with a 92.48% classification accuracy rate. The CNN-CAD results showed a better performance in all criteria than those of endoscopic experts. The model visualization results showed reasonable regions of interest to explain pathology classification decisions. We demonstrated that CNN-CAD can distinguish the pathology of colorectal adenoma, yielding better outcomes than the endoscopic experts group.