Clinical insights to improve medical deep learning design: A comprehensive review of methods and benefits

The success of deep learning and computer vision of natural images has led to an increased interest in medical image deep learning applications. However, introducing black-box deep learning models leaves little room for domain-specific knowledge when making the final diagnosis. For medical computer vision applications, not only accuracy, but also robustness, interpretability and explainability are essential to ensure trust for clinicians. Medical deep learning applications can therefore benefit from insights into the application at hand by involving clinical staff and considering the clinical diagnostic process. In this review, different clinically-inspired methods are surveyed, including clinical insights used at different stages of deep learning design for three-dimensional (3D) computed tomography (CT) image data. This review is conducted by investigating 400 research articles, covering different deep learning-based approaches for diagnosis of different diseases, in terms of including clinical insights in the published work. Based on this, a further detailed review is conducted of the 47 scientific articles using clinical inspiration. The clinically-inspired methods were found to be made with respect to preparation for training, 3D medical image data processing, integration of clinical data and model architecture selection and development. This highlights different ways in which domain-specific knowledge can be used in the design of deep learning systems.