External validation of an algorithm to detect vertebral level mislabeling and autocontouring errors.

Area of Science:

• Medical Imaging
• Artificial Intelligence in Radiology
• Machine Learning for Medical Diagnosis

Background:

• Automated contouring tools for vertebral bodies are essential for efficient radiological analysis.
• External validation is critical to ensure the reliability of AI tools across different healthcare institutions.
• Previous vertebral body autocontouring tools require further performance enhancement for clinical use.

Purpose of the Study:

• To externally validate a developed vertebral body autocontouring tool.
• To investigate a post-processing method to enhance the tool's performance to clinically acceptable levels.
• To assess the tool's performance metrics including identification rate, contour acceptability, and quality assurance accuracy.

Main Methods:

• External validation using CT scans from two institutions (40 from A, 41 from B).
• Automatic localization, enumeration, contouring, and quality assurance screening of vertebral bodies (C1-L5).
• Comparison of performance metrics against the original training dataset and development of a post-processing technique.

Main Results:

• Initial testing showed identification rates of 83% (A) and 92% (B), with reduced performance compared to the training dataset.
• Post-processing adjustment significantly increased identification rates by an average of 4% for both datasets (p < 0.01).
• The adjusted algorithm demonstrated improved accuracy for vertebral body localization.

Conclusions:

• A post-processing adjustment in the machine learning pipeline enhanced vertebral body localization accuracy to clinically acceptable standards.
• External validation of machine learning and deep learning tools is imperative prior to deployment in diverse clinical environments.
• The study highlights the importance of rigorous validation for AI-driven medical imaging tools.