Improving Clinical Utility of Real-World Prediction Models: Updating Through Recalibration

Area of Science:

• Sports Medicine
• Biostatistics
• Baseball Performance Analysis

Background:

• Prediction models assist clinicians in identifying at-risk athletes.
• Evolving sport and clinical practices can lead to predictive drift and reduced model performance.
• Temporal recalibration is necessary for previously developed baseball arm injury models.

Purpose of the Study:

• To perform temporal recalibration on a previously developed baseball arm injury prediction model.
• To assess the performance of the recalibrated model in professional baseball pitchers.
• To compare the clinical utility of the recalibrated model against existing practices.

Main Methods:

• An arm injury prediction model was developed using data from minor league pitchers (2009-2019).
• Data from 2015-2019 seasons were used for temporal recalibration (intercept-only and full redevelopment).
• Model performance was evaluated using Nagelkerke's R-square, calibration, and discrimination metrics.

Main Results:

• The temporal recalibrated intercept model showed the best discrimination (0.81) and R-square (0.32).
• This model outperformed the original model (discrimination: 0.74) and the redeveloped model (discrimination: 0.80).
• The recalibrated intercept model demonstrated a 0.34 improved net benefit over current best evidence-based practice.

Conclusions:

• Temporal recalibration enhances the performance and clinical utility of baseball arm injury prediction models.
• Updated models can effectively account for changes in sport training over time.
• Recalibration can lead to improved arm injury outcomes for professional baseball pitchers.