External Validation of Two AI Systems for Dentinal Caries Detection: A Retrospective Pilot Clinical Outcomes Study

To evaluate the sensitivity of two dental artificial intelligence (AI) systems for detecting dentinal caries using longitudinal, outcomes-anchored validation within an integrated healthcare system.

In this pilot external validation study, two commercially available AI systems were tested on full-mouth intraoral radiographic series, including bitewings, from 90 patients in a federal healthcare system. Ground truth was established using baseline clinical examinations and follow-up documentation 6-12 months later. Vendor-specific and pooled sensitivity for dentinal caries detection were calculated.

Vendor A achieved a sensitivity of 48.5% (95% CI: 37.0-60.2), Vendor B achieved 57.5% (95% CI: 46.1-68.2), and pooled sensitivity was 53.0% (95% CI: 44.8-61.1).

Outcomes-anchored, longitudinal validation of dental AI systems for dentinal caries detection was feasible within a large integrated healthcare system. This approach provides clinically relevant sensitivity estimates grounded in real treatment outcomes. These findings are most directly applicable to large integrated healthcare systems with standardized longitudinal data capture and may not fully reflect the variability of private-practice dental settings.

This study establishes a reproducible framework for evaluating AI diagnostic performance in real-world healthcare settings using longitudinal clinical outcomes rather than radiographic labels alone, demonstrating that independent validation with clinical follow-up is feasible within a large healthcare system.