Assessing discriminative ability of risk models in clustered data

Area of Science:

• Biostatistics
• Epidemiology
• Health Services Research

Background:

• Risk models are vital for clinical decision-making, with Harrell's concordance-index (c-index) measuring discriminative ability.
• In clustered data (e.g., multicenter studies), within-cluster and between-cluster concordance are distinguished.
• Within-cluster concordance is most relevant for risk models guiding decisions at the local (cluster) level.

Purpose of the Study:

• To explore and compare different approaches for estimating the within-cluster concordance probability in clustered data.
• To identify the most suitable meta-analytical techniques for pooling cluster-specific concordance indexes.

Main Methods:

• Utilized data from the CRASH trial (2,081 patients, 35 centers) to develop a risk model for mortality after traumatic brain injury.
• Calculated cluster-specific c-indexes to assess model discrimination within each center.
• Applied various meta-analytical techniques, including fixed and random effects models, to pool cluster-specific c-indexes.

Main Results:

• Cluster-specific c-indexes showed substantial variation across centers (IQR: 0.70-0.81).
• Fixed effect meta-analysis yielded summary estimates ranging from 0.75 to 0.84.
• Random effects meta-analysis estimated a mean c-index of 0.77 with a 95% prediction interval of 0.60 to 0.95, accounting for heterogeneity.

Conclusions:

• Meta-analysis of cluster-specific c-indexes is recommended for evaluating risk models that support cluster-level decisions.
• Random effects meta-analysis is particularly advised due to observed heterogeneity in discriminative ability across clusters.