Generalizability of Risk Stratification Algorithms for Exacerbations in COPD

Area of Science:

• Pulmonary Medicine
• Clinical Epidemiology
• Biostatistics

Background:

• Current COPD management uses exacerbation history for risk stratification.
• Multivariable prediction models can enhance risk stratification accuracy.
• The clinical utility of risk stratification tools can differ across patient populations.

Purpose of the Study:

• To compare the performance of two validated exacerbation risk prediction models, the Acute COPD Exacerbation Prediction Tool (ACCEPT) and the Bertens model, against exacerbation history alone.
• To evaluate model performance across different patient populations with varying exacerbation risk levels.
• To assess the impact of model recalibration on clinical utility.

Main Methods:

• Utilized data from three clinical studies (SUMMIT, LOTT, TORCH) with diverse COPD exacerbation risk levels.
• Compared risk stratification algorithms using Area Under the Receiver Operating Characteristic Curve (AUC) and net benefit.
• Evaluated the effect of model recalibration on the clinical utility of prediction models.

Main Results:

• Both ACCEPT and the Bertens model demonstrated superior predictive performance compared to exacerbation history alone across most samples.
• No single algorithm consistently outperformed others in clinical utility across all populations.
• Recalibration substantially mitigated the risk of harm associated with using prediction models, enhancing their clinical utility.

Conclusions:

• Exacerbation history alone has limited clinical utility and potential risk of harm for predicting COPD exacerbations.
• Validated prediction models offer improved predictive performance but necessitate setting-specific recalibration for optimal clinical utility.
• Recalibration is crucial for maximizing the benefit and minimizing the harm of COPD exacerbation prediction models.