Rethinking Diagnostic Performance Metrics in Imaging Artificial Intelligence: Lessons from Caries Detection on...

Area of Science:

• Radiology
• Artificial Intelligence
• Dental Diagnostics

Background:

• Artificial intelligence (AI) systems are increasingly used for radiographic caries detection.
• Current evaluations often rely on limited performance metrics lacking clinical context.
• This leads to challenges in interpreting and comparing AI system performance.

Purpose of the Study:

• To review and discuss essential performance metrics for AI in radiographic caries detection.
• To highlight the importance of contextual information for meaningful interpretation of AI performance.
• To provide guidance on transparent reporting for improved comparability and transportability of AI systems.

Main Methods:

• Narrative review focusing on confusion-matrix and discrimination metrics.
• Discussion of complementary assessments: calibration, decision-analytic metrics, and robustness.
• Analysis of how interpretation depends on disease definition, unit of analysis, thresholds, and prevalence.

Main Results:

• Performance metrics like sensitivity, specificity, accuracy, ROC-AUC, and PR-AUC require explicit reporting of clinical context.
• Calibration, decision curves, and robustness assessments offer complementary insights.
• Lack of standardized reporting hinders clinical utility and transportability of AI tools.

Conclusions:

• Clinically meaningful evaluation of AI for radiographic caries detection demands comprehensive reporting beyond standard metrics.
• Transparent reporting of disease definitions, thresholds, prevalence, and complementary assessments is crucial.
• Standardized reporting frameworks will enhance the reliability and clinical adoption of AI in dentistry.