Skin lesion classification with ensembles of deep convolutional neural networks

Skin cancer is a major public health problem with over 123,000 newly diagnosed cases worldwide in every year. Melanoma is the deadliest form of skin cancer, responsible for over 9000 deaths in the United States each year. Thus, reliable automatic melanoma screening systems would provide a great help for clinicians to detect the malignant skin lesions as early as possible. In the last five years, the efficiency of deep learning-based methods increased dramatically and their performances seem to outperform conventional image processing methods in classification tasks. However, this type of machine learning-based approaches have a main drawback, namely they require thousands of labeled images per classes for their training. In this paper, we investigate how we can create an ensemble of deep convolutional neural networks to improve further their individual accuracies in the task of classifying dermoscopy images into the three classes melanoma, nevus, and seborrheic keratosis when we have no opportunity to train them on adequate number of annotated images. To achieve high classification accuracy, we fuse the outputs of the classification layers of four different deep neural network architectures. More specifically, we propose the aggregation of robust convolutional neural networks (CNNs) into one framework, where the final classification is achieved based on the weighted output of the member CNNs. For aggregation, we consider different fusion-based methods and select the best performing one for this problem. Our experimental results also prove that the creation of an ensemble of different neural networks is a meaningful approach, since each of the applied fusion strategies outperforms the individual networks regarding classification accuracy. The average area under the receiver operating characteristic curve has been found to be 0.891 for the 3-class classification task. For an objective evaluation of our approach, we have tested its performance on the official test database of the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 challenge on Skin Lesion Analysis Towards Melanoma Detection dedicated to skin cancer recognition.