Net reclassification indices for evaluating risk prediction instruments: a critical review

Area of Science:

• Biostatistics
• Medical Informatics
• Epidemiology

Background:

• Net reclassification indices (NRIs) are increasingly used to assess the predictive improvement of new biomarkers.
• Accurate interpretation and application of NRIs are crucial for reliable clinical decision-making.
• Existing measures for prediction increment require careful evaluation alongside novel indices like NRIs.

Purpose of the Study:

• To review and critically interpret various types of net reclassification indices.
• To evaluate the advantages and disadvantages of using NRIs for quantifying prediction increments.
• To compare NRIs with existing measures and provide recommendations for their appropriate use.

Main Methods:

• Review of existing literature on net reclassification indices and related statistical measures.
• Comparative analysis of NRIs against established metrics like true positive rates and false positive rates.
• Evaluation of statistical methodologies for confidence intervals and hypothesis testing related to NRIs.

Main Results:

• For two risk categories, NRIs are equivalent to changes in true and false positive rates.
• NRIs are not recommended for three or more risk categories due to inadequate accounting for risk shifts.
• The category-free NRI can overstate biomarker value and shares limitations with measures like AUC.

Conclusions:

• Investigators should report NRIs separately for cases and controls.
• True and false positive rates are preferred over NRIs for clarity and interpretability.
• Improvement in net benefit is the recommended single-number summary for prediction increment; bootstrap methods are preferred for confidence intervals.