Guide to evaluating performance of prediction models for recurrent clinical events

Area of Science:

• Biostatistics
• Clinical Epidemiology
• Chronic Disease Management

Background:

• Chronic conditions like epilepsy and asthma are characterized by recurrent events.
• Traditional statistical models often focus only on the time to the first event, neglecting subsequent events.
• Effective statistical models for recurrent events are crucial for accurate patient outcome prediction.

Purpose of the Study:

• To compare the performance of different statistical models for analyzing recurrent events in epilepsy and asthma.
• To evaluate the effectiveness of various models in predicting the risk of recurrent exacerbations and seizures.
• To identify optimal methods for assessing the performance of recurrent event prediction models.

Main Methods:

• Utilized two clinical datasets: asthma exacerbations and epileptic seizures.
• Applied count-based models (negative binomial, zero-inflated negative binomial) and Cox model variants (Andersen-Gill, Prentice, Williams and Peterson).
• Evaluated model performance using numerical (RMSE, MAE, bias) and graphical (calibration plots, Bland-Altman plots) approaches.

Main Results:

• Model performance for recurrent asthma and epilepsy events was assessed using numerical and graphical measures.
• The Prentice, Williams and Peterson model exhibited the highest agreement between predicted and observed outcomes for both asthma and epilepsy.
• This indicates its superior ability to model recurrent events in these chronic conditions.

Conclusions:

• Inappropriate statistical models can lead to erroneous conclusions, potentially harming patients.
• Prediction models for chronic conditions must incorporate all recurrent events for accurate risk assessment.
• Recommended numerical and graphical approaches, along with modified calibration measures, for evaluating these models.