Evaluating the Robustness of a Deep Learning Bone Age Algorithm to Clinical Image Variation Using Computational...

Area of Science:

• Medical imaging and artificial intelligence
• Pediatric radiology
• Deep learning in healthcare

Background:

• Deep learning (DL) models are increasingly used for medical image analysis.
• Evaluating model robustness to real-world image variations is crucial for clinical adoption.
• Bone age assessment is a common task in pediatric radiology.

Purpose of the Study:

• To assess the robustness of a leading pediatric bone age DL model.
• To evaluate performance against image appearance variations simulating real-world conditions.
• To determine if the model's accuracy is affected by common image transformations.

Main Methods:

• Retrospective evaluation of a 2017 RSNA Pediatric Bone Age Challenge winning DL model.
• Testing on two datasets: RSNA validation set and Digital Hand Atlas (DHA).
• Applying seven image transformations (rotations, flips, brightness, contrast, inversion, laterality marker, resolution) to simulate variations.

Main Results:

• The DL model generalized well to the external DHA dataset (MAD 6.9 months vs. RSNA 6.8 months).
• Significant differences in mean absolute differences (MAD) were observed for most transformations on both datasets.
• Clinically significant errors increased for 57% of transformations on the DHA dataset.

Conclusions:

• The pediatric bone age DL model demonstrated good generalization to external data.
• The model exhibited inconsistent predictions when subjected to common image transformations.
• Robustness to real-world image variations remains a challenge for current pediatric bone age DL models.