Real-World Performance of Large Vessel Occlusion Artificial Intelligence-Based Computer-Aided Triage and Notification...

Area of Science:

• Neurology
• Medical Imaging
• Artificial Intelligence

Background:

• Large-vessel occlusion (LVO) is a critical factor in stroke outcomes.
• AI-based computer-aided triage and notification (CADt) devices are FDA-approved to aid in LVO detection.
• Evaluating real-world performance is crucial for understanding clinical utility.

Purpose of the Study:

• To assess the real-world performance of two FDA-approved AI-based CADt devices for LVO detection.
• To compare this real-world performance against manufacturer-reported data.

Main Methods:

• Retrospective evaluation of CT angiography examinations from two stroke centers.
• Inclusion of consecutive "code stroke" cases.
• Assessment of LVO in multiple intracranial and proximal vessel segments, with radiology reports serving as the reference standard.

Main Results:

• At Hospital A, real-world sensitivity/specificity for ICA/M1 segments were 85.3%/91.9%, decreasing to 59.9% for all proximal segments.
• At Hospital B, real-world sensitivity/specificity for ICA/M1 segments were 90.7%/97.9%, decreasing to 59.4% for all proximal segments.
• Manufacturer data reported higher sensitivities and specificities (e.g., 97%/95.6% at Hospital A for ICA/MCA).

Conclusions:

• Real-world performance of CADt LVO detection algorithms shows significant gaps compared to manufacturer claims.
• Detection and communication of treatable LVOs are compromised when considering vessels beyond ICA/M1 segments.
• Performance is further impacted by absent or uninterpretable data.