Problems with risk reclassification methods for evaluating prediction models.

Area of Science:

• Biostatistics
• Epidemiology
• Medical Informatics

Background:

• Risk prediction models are crucial for clinical decision-making.
• Evaluating improvements in risk prediction models requires robust statistical methods.
• Current risk reclassification analysis strategies have limitations in interpreting performance.

Purpose of the Study:

• To critically evaluate a proposed risk reclassification analysis strategy for comparing prediction models.
• To identify limitations in the interpretation of summary measures and P values within this strategy.
• To recommend alternative approaches for assessing model performance.

Main Methods:

• Cross-classification of risks from baseline and enhanced prediction models.
• Calculation of summary measures like percentage of reclassification and correct reclassification.
• Assessment of proposed reclassification calibration statistics.

Main Results:

• Interpretations of proposed summary measures and P values in risk reclassification analysis are problematic.
• The reclassification table provides valuable information but is insufficient alone for model comparison.
• Alternative methods, such as the Net Reclassification Index, are suggested for improved performance assessment.

Conclusions:

• The proposed risk reclassification analysis strategy has interpretational issues.
• Displaying reclassification tables is informative but requires complementary analytical methods.
• Reporting components of the Net Reclassification Index offers more clinically relevant insights than a single summary measure.