Computer-aided diagnosis system for breast ultrasound images using deep learning

The purpose of this study was to develop a computer-aided diagnosis (CAD) system for the classification of malignant and benign masses in the breast using ultrasonography based on a convolutional neural network (CNN), a state-of-the-art deep learning technique. We explored the regions for the correct classification by generating a heat map that presented the important regions used by the CNN for human malignancy/benign classification. Clinical data was obtained from a large-scale clinical trial previously conducted by the Japan Association of Breast and Thyroid Sonology. Images of 1536 breast masses (897 malignant and 639 benign) confirmed by pathological examinations were collected, with each breast mass captured from various angles using an ultrasound (US) imaging probe. We constructed an ensemble network by combining two CNN models (VGG19 and ResNet152) fine-tuned on balanced training data with augmentation and used the mass-level classification method to enable the CNN to classify a given mass using all views. For an independent test set consisting of 154 masses (77 malignant and 77 benign), our network showed outstanding classification performance with a sensitivity of 90.9% (95% confidence interval 84.5-97.3), a specificity of 87.0% (79.5-94.5), and area under the curve (AUC) of 0.951 (0.916-0.987) compared to that of the two CNN models. In addition, our study indicated that the breast masses themselves were not detected by the CNN as important regions for correct mass classification. Collectively, this CNN-based CAD system is expected to assist doctors by improving the diagnosis of breast cancer in clinical practice.