Content-based Image Retrieval by Using Deep Learning for Interstitial Lung Disease Diagnosis with Chest CT

Background Evaluation of interstitial lung disease (ILD) at CT is a challenging task that requires experience and is subject to substantial interreader variability. Purpose To investigate whether a proposed content-based image retrieval (CBIR) of similar chest CT images by using deep learning can aid in the diagnosis of ILD by readers with different levels of experience. Materials and Methods This retrospective study included patients with confirmed ILD after multidisciplinary discussion and available CT images identified between January 2000 and December 2015. Database was composed of four disease classes: usual interstitial pneumonia (UIP), nonspecific interstitial pneumonia (NSIP), cryptogenic organizing pneumonia, and chronic hypersensitivity pneumonitis. Eighty patients were selected as queries from the database. The proposed CBIR retrieved the top three similar CT images with diagnosis from the database by comparing the extent and distribution of different regional disease patterns quantified by a deep learning algorithm. Eight readers with varying experience interpreted the query CT images and provided their most probable diagnosis in two reading sessions 2 weeks apart, before and after applying CBIR. Diagnostic accuracy was analyzed by using McNemar test and generalized estimating equation, and interreader agreement was analyzed by using Fleiss κ. Results A total of 288 patients were included (mean age, 58 years ± 11 [standard deviation]; 145 women). After applying CBIR, the overall diagnostic accuracy improved in all readers (before CBIR, 46.1% [95% CI: 37.1, 55.3]; after CBIR, 60.9% [95% CI: 51.8, 69.3]; P < .001). In terms of disease category, the diagnostic accuracy improved after applying CBIR in UIP (before vs after CBIR, 52.4% vs 72.8%, respectively; P < .001) and NSIP cases (before vs after CBIR, 42.9% vs 61.6%, respectively; P < .001). Interreader agreement improved after CBIR (before vs after CBIR Fleiss κ, 0.32 vs 0.47, respectively; P = .005). Conclusion The proposed content-based image retrieval system for chest CT images with deep learning improved the diagnostic accuracy of interstitial lung disease and interreader agreement in readers with different levels of experience. © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Wielpütz in this issue.