Related Experiment Video
Updated: Jun 11, 2025
Brain Infarct Segmentation and Registration on MRI or CT for Lesion-symptom Mapping
Published on: September 25, 2019
Boosting Deep Learning for Interpretable Brain MRI Lesion Detection through the Integration of Radiology Report
Lisong Dai1, Jiayu Lei1, Fenglong Ma1
1From the Institute of Diagnostic and Interventional Radiology (L.D., Z.S., H.D., J.J., D.W., G.T., X.S., J.Z., Q.Z., Y.L.) and Clinical Research Center (J.W.), Shanghai Sixth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, 600 Yishan Road, Shanghai 200000, China; Shanghai AI Laboratory, Shanghai, China (J.L., Y.Z.); School of Computer Science and Technology, University of Science and Technology of China, Anhui, China (J.L.); The Pennsylvania State University College of Information Sciences and Technology, University Park, Pa (F.M.); Department of Electrical Engineering, City University of Hong Kong, Hong Kong, China (H.Z.); Department of Radiology, Affiliated Hospital of Nantong University, Nantong, China (J.J.); Department of Radiology, Shanghai Ninth People's Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China (S.A.); Department of Radiology, Shanghai Public Health Clinical Center, Shanghai, China (A.S.); Department of Radiology, Wuhan Hankou Hospital, Wuhan, China (Z.L.); and Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai, China (Y.Z.).
None:
Purpose To guide the attention of a deep learning (DL) model toward MRI characteristics of brain lesions by incorporating radiology report-derived textual features to achieve interpretable lesion detection. Materials and Methods In this retrospective study, 35 282 brain MRI scans (January 2018 to June 2023) and corresponding radiology reports from center 1 were used for training, validation, and internal testing. A total of 2655 brain MRI scans (January 2022 to December 2022) from centers 2-5 were reserved for external testing. Textual features were extracted from radiology reports to guide a DL model (ReportGuidedNet) focusing on lesion characteristics. Another DL model (PlainNet) without textual features was developed for comparative analysis. Both models identified 15 conditions, including 14 diseases and normal brains. Performance of each model was assessed by calculating macro-averaged area under the receiver operating characteristic curve (ma-AUC) and micro-averaged AUC (mi-AUC). Attention maps, which visualized model attention, were assessed with a five-point Likert scale. Results ReportGuidedNet outperformed PlainNet for all diagnoses on both internal (ma-AUC, 0.93 [95% CI: 0.91, 0.95] vs 0.85 [95% CI: 0.81, 0.88]; mi-AUC, 0.93 [95% CI: 0.90, 0.95] vs 0.89 [95% CI: 0.83, 0.92]) and external (ma-AUC, 0.91 [95% CI: 0.88, 0.93] vs 0.75 [95% CI: 0.72, 0.79]; mi-AUC, 0.90 [95% CI: 0.87, 0.92] vs 0.76 [95% CI: 0.72, 0.80]) testing sets. The performance difference between internal and external testing sets was smaller for ReportGuidedNet than for PlainNet (Δma-AUC, 0.03 vs 0.10; Δmi-AUC, 0.02 vs 0.13). The Likert scale score of ReportGuidedNet was higher than that of PlainNet (mean ± SD: 2.50 ± 1.09 vs 1.32 ± 1.20; P < .001). Conclusion The integration of radiology report textual features improved the ability of the DL model to detect brain lesions, thereby enhancing interpretability and generalizability. Keywords: Deep Learning, Computer-aided Diagnosis, Knowledge-driven Model, Radiology Report, Brain MRI Supplemental material is available for this article. Published under a CC BY 4.0 license.
More Related Videos
12:50Lesion Explorer: A Video-guided, Standardized Protocol for Accurate and Reliable MRI-derived Volumetrics in Alzheimer's Disease and Normal Elderly
Published on: April 14, 2014
04:48Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography
Published on: November 30, 2022
