External validation of predictive models for diagnosis, management and severity of pediatric appendicitis

Area of Science:

• Medical Informatics
• Pediatric Surgery
• Artificial Intelligence in Healthcare

Background:

• Appendicitis is a frequent condition in children and adolescents.
• Machine learning (ML) models can aid in diagnosing, assessing severity, and guiding management of pediatric appendicitis.
• Model reliability, safety, and robustness are crucial for adoption across diverse clinical settings.

Purpose of the Study:

• To externally validate ML models for pediatric appendicitis diagnosis, management, and severity prediction.
• To assess the impact of differing hospital practices and patient populations on model performance.
• To evaluate the effectiveness of model retraining for improving performance in new environments.

Main Methods:

• External validation of ML models trained on a German cohort (430 patients) using an independent German cohort (301 patients).
• Inclusion of demographic, clinical, scoring, laboratory, and ultrasound parameters.
• Exploration of model retraining benefits and variable importance analysis.

Main Results:

• Significant differences in parameter distributions between datasets led to decreased predictive performance for diagnosis, management, and severity.
• Retraining models with external data improved performance, but it remained lower than the original study's performance.
• Key predictive variables demonstrated consistency across both hospital datasets.

Conclusions:

• Transferred ML models for pediatric appendicitis achieved satisfactory performance but were suboptimal compared to original data performance.
• Transferring models between hospitals presents challenges due to variations in clinical practices, demographics, and external factors.
• Model retraining offers partial improvement but cannot fully recover performance lost during transfer, indicating limitations in its remedial capacity.