Non-transfer Deep Learning of Optical Coherence Tomography for Post-hoc Explanation of Macular Disease Classification

Deep transfer learning is a popular choice for classifying monochromatic medical images using models that are pretrained by natural images with color channels. This choice may introduce unnecessarily redundant model complexity that can limit explanations of such model behavior and outcomes in the context of medical imaging. To investigate this hypothesis, we develop a configurable deep convolutional neural network (CNN) to classify four macular disease conditions using retinal optical coherence tomography (OCT) images. Our proposed non-transfer deep CNN model (acc: 97.9%) outperforms existing transfer learning models such as ResNet-50 (acc: 89.0%), ResNet-101 (acc: 96.7%), VGG-19 (acc: 93.3%), Inception-V3 (acc: 95.8%) in the same retinal OCT image classification task. We perform post-hoc analysis of the trained model and model extracted image features, which reveals that only eight out of 256 filter kernels are active at our final convolutional layer. The convolutional responses of these selective eight filters yield image features that efficiently separate four macular disease classes even when projected onto two-dimensional principal component space. Our findings suggest that many deep learning parameters and their computations are redundant and expensive for retinal OCT image classification, which are expected to be more intense when using transfer learning. Additionally, we provide clinical interpretations of our misclassified test images identifying manifest artifacts, shadowing of useful texture, false texture representing fluids, and other confounding factors. These clinical explanations along with model optimization via kernel selection can improve the classification accuracy, computational costs, and explainability of model outcomes.