Prediction models for clustered data: comparison of a random intercept and standard regression model

Area of Science:

• Biostatistics
• Health Services Research
• Clinical Prediction Modeling

Background:

• Standard regression analysis is unsuitable for clustered data; specialized techniques are necessary.
• Random effect regression models are preferred for patient-level predictor effect research.
• Parameter estimates differ between random effect and standard logistic regression models.

Purpose of the Study:

• To compare the predictive accuracy of random effect and standard logistic regression models.
• To evaluate model performance in clustered data settings.
• To assess external validity and calibration of prognostic models.

Main Methods:

• Developed prognostic models using standard and random intercept logistic regression on 1642 surgical patients.
• Assessed external validity in new patients treated by different anesthesiologists (clusters).
• Conducted simulation studies with varying intra-class correlation coefficients (ICC) and estimated performance measures.

Main Results:

• Random effect models demonstrated superior discrimination when cluster effects were used for prediction (c-index 0.69 vs. 0.66).
• External validation showed similar discrimination for both models (c-index 0.68 vs. 0.67).
• Model calibration was adequate for external subjects with high ICC (≥15%) only when performance measures matched the model development method.

Conclusions:

• Random intercept models offer better discrimination than standard models when cluster effects are utilized for prediction.
• The random intercept prediction model exhibited good within-cluster calibration.
• Choosing appropriate performance measures is crucial for accurate calibration in clustered data analysis.