Reliability and diagnostic performance of an automated MRI-based classifier compared with radiologists in Alzheimer's...

Reliable imaging biomarkers are essential for improving early detection of Alzheimer's disease (AD). We evaluated whether an automated MRI-based classifier provides diagnostic performance comparable to expert radiologists in differentiating cognitively normal (CN) individuals from patients with AD using standardized ADNI data. Thirty-eight structural MRI datasets (20 CN, 18 AD) were analyzed. An automated multi-class volumetric classifier and two board-certified radiologists independently assigned probability scores across seven diagnostic categories. Performance was evaluated using a partial-credit scoring rule to account for probabilistic ties. Diagnostic performance for CN-AD discrimination was assessed using accuracy, sensitivity, specificity, receiver operating characteristic (ROC) analysis, inter-observer agreement metrics, Brier scores for calibration, and decision curve analysis (DCA) for clinical utility. The automated classifier achieved an accuracy of 0.66, sensitivity of 0.56, and specificity of 0.75. Radiologists demonstrated comparable performance with inherent inter-observer variability. Agreement between automated and human assessments was fair at the categorical level, with low concordance for continuous probability estimates. ROC analysis based on continuous AD probabilities demonstrated high discrimination performance for the automated model (AUC = 0.90), exceeding that of radiologists (AUC = 0.71 and 0.62). DCA indicated that the automated pipeline provides a positive net benefit as a second-opinion tool. This exploratory study emphasizes the impact of evaluation frameworks on performance metrics and supports further validation using multi-modal data in larger cohorts.