Evaluating Deep Learning-Based Commercial Software for Detecting Ischemic Lesions on DWI in Stroke Patients

Background: Recent advancements in deep learning have enabled the development of automated software to assist in ischemic lesion detection on diffusion-weighted imaging (DWI), but their real-world performance remains underexplored. This study evaluated the diagnostic performance of a commercially available, CE-marked (MDR class IIa) artificial intelligence (AI) software version 1.0 for detecting ischemic lesions on DWI and examined its sensitivity in relation to lesion-specific characteristics. Methods: A retrospective cohort of 235 patients with confirmed ischemic stroke who underwent DWI was analyzed. The CE-marked software's performance was assessed at both lesion and patient-level, using expert neurologist interpretations as the reference standard. Lesion characteristics, including maximum axial size, apparent diffusion coefficient (ADC) values, slice coverage, and anatomical location, were analyzed. Results: The software achieved a lesion-level sensitivity of 83.51% (95% CI, 79.8-86.8%) and a patient-level sensitivity of 95.31% (95% CI, 91.8-97.6%). Undetected lesions were significantly smaller, covered fewer slices, and had higher ADC values. No significant differences were observed in detection rates by anatomical locations, vascular territories, or time from symptom onset. Conclusions: While the AI software demonstrated a strong patient-level sensitivity overall, it showed limitations in identifying smaller, less conspicuous lesions. These findings underscore the need to optimize deep learning algorithms for better sensitivity and highlight the importance of clinician awareness regarding AI limitations in acute stroke care.