Diagnostic Performance Evaluation of a Computer-Assisted Imaging Analysis System for Ultrasound Risk Stratification...

OBJECTIVE. Ultrasound-based stratification of the malignancy risk of thyroid nodules has potential variability. The purpose of this study is to evaluate the diagnostic effectiveness of the first commercially available system for computer-aided diagnosis (CADx) imaging analysis. MATERIALS AND METHODS. Ultrasound images of 300 thyroid nodules (135 of which were malignant) acquired before surgical treatment were retrospectively reviewed by a thyroid expert, and his classification of each image was then compared with the classification rendered by an image analysis program (AmCAD-UT, AmCAD Biomed). The American Thyroid Association (ATA) classification system, the European Thyroid Imaging Reporting and Data System (EU-TIRADS), and the classification system jointly proposed by American and Italian associations of clinical endocrinologists (the American Association of Clinical Endocrinologists [AACE], the American College of Endocrinology [ACE], and Associazione Medici Endocrinologi [AME]) were used for risk stratification. RESULTS. The diagnostic performance of the thyroid expert when the ATA system was used was as follows: sensitivity, 87.0%; specificity, 91.2%; positive predictive value, 90.5%; and negative predictive value, 90.9%. Compared with the expert, the CADx program, when used with the three classification systems, had a similar sensitivity but a lower specificity and positive predictive value. Regarding the negative predictive value, the results of the expert did not differ from those of the CADx program when it applied the ATA classification system (90.9% vs 86.3%; p = 0.07). The ROC AUC value was 0.88 for the expert clinician and 0.72 for the CADx program when the ATA classification system was used. CONCLUSION. The CADx ultrasound image analysis program described in the present study is useful for risk stratification of thyroid nodules, but it does not perform better than a sonography expert.