Multi-Center Validation of Artificial Intelligence-Based Video Analysis Platform for Automatic Evaluation of...

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Swallowing Disorders

Background:

• Videofluoroscopic swallow study (VFSS) is crucial for assessing swallowing function.
• Existing artificial intelligence (AI) models for VFSS lack extensive external validation.
• This study addresses the need for robust AI evaluation in VFSS interpretation.

Purpose of the Study:

• To develop an AI model for automatic diagnosis of aspiration and penetration from VFSS videos.
• To evaluate the AI model's performance through multicenter external validation.
• To assess the clinical utility of AI in swallowing disorder diagnostics.

Main Methods:

• Developed an AI model using YOLOv9_c architecture on 2012 VFSS images for training.
• Internally validated the model on 309 Q1-grade VFSS videos.
• Conducted external validation using 138 Q1 and Q2-grade VFSS videos from two hospitals.

Main Results:

• Internal validation showed high training (98.1%) and validation (97.8%) accuracy, with test accuracy at 61.5%.
• AI model achieved diagnostic accuracy for aspiration (AUC=0.70) and penetration (AUC=0.65) in internal testing.
• External validation demonstrated strong performance: aspiration (precision 90.2%, AUC=0.79) and penetration (precision 78.3%, AUC=0.80), with overall accuracy of 80.4%.

Conclusions:

• An AI model was successfully developed for automatic aspiration and penetration diagnosis from VFSS videos.
• Multicenter external validation confirmed the AI model's good diagnostic accuracy.
• Further research with larger, diverse datasets is needed to enhance clinical application.