Predicting radiology no-shows with machine learning: Methodological pitfalls and practical considerations.

Area of Science:

• Health Informatics
• Machine Learning in Healthcare
• Operational Efficiency

Background:

• Radiology appointment no-shows and late cancellations disrupt patient care and cause financial losses.
• Developing robust no-show prediction models in large health systems presents significant practical challenges.

Purpose of the Study:

• To outline the methodological hurdles encountered in building outpatient radiology no-show prediction models at a large academic medical center.
• To identify key predictors and data-related challenges in developing reliable no-show prediction models.

Main Methods:

• Utilized a dataset of 334,002 outpatient imaging appointments, integrating EHR data with external weather, socioeconomic, and geographic information.
• Employed preprocessing techniques for cohort selection, feature engineering, and addressing class imbalance and data leakage.
• Trained boosted tree classifiers and other algorithms on 2022-2023 data and tested on 2024 data.

Main Results:

• The best models achieved an Area Under the Curve (AUC) of 0.71.
• Identified challenges including ambiguous data definitions, distinguishing cancellations from reschedules, and variable data quality.
• Found phone reminder status, appointment confirmation, and modality to be key predictors, while crime, income, and weather had limited utility.

Conclusions:

• Building effective no-show prediction models necessitates extensive data cleaning, feature engineering, and temporal validation.
• Real-world data constraints pose risks of flawed models due to leakage, inconsistencies, and confounding factors.
• Successful deployment requires transparent documentation, rigorous validation, and critical data quality assessments beyond algorithmic sophistication.