Validation of an Automated ASPECTS Software via a Multi-Reader Multi-Case Clinical Reader Study

Area of Science:

• Radiology
• Artificial Intelligence
• Neurology

Background:

• Early detection of ischemic changes on non-contrast CT is crucial for acute ischemic stroke (AIS) patient triage.
• Interpreting non-contrast CT scans for early ischemic changes (EIC) is challenging due to low contrast and reader variability.

Purpose of the Study:

• To validate StrokeSENS ASPECTS, an AI tool for automated grading of EIC on NCCT.
• To assess the impact of StrokeSENS ASPECTS on clinician performance and inter-reader agreement.

Main Methods:

• A multi-reader, multi-case study involving eight clinicians evaluating 100 NCCT scans.
• Clinicians interpreted scans unaided and aided by StrokeSENS ASPECTS.
• Performance was measured using binary classification metrics and inter-reader agreement (Fleiss's kappa).

Main Results:

• StrokeSENS ASPECTS improved balanced accuracy by 5.7%, overall accuracy by 2.6%, sensitivity by 9.7%, and specificity by 1.6%.
• Inter-reader agreement significantly increased, with Fleiss's kappa rising from 0.323 (unaided) to 0.608 (aided).

Conclusions:

• StrokeSENS ASPECTS enhances clinicians' ability to detect EIC on NCCT.
• The AI tool effectively reduces inter-reader variability, demonstrating safety and effectiveness for AIS evaluation.