Evaluating a new marker for risk prediction: decision analysis to the rescue

Area of Science:

• Medical Statistics
• Clinical Decision Making
• Risk Prediction Modeling

Background:

• Risk prediction models are crucial in medicine for identifying individuals who may benefit from treatment.
• Current evaluation of new risk prediction markers often relies solely on predictive accuracy metrics.
• Existing accuracy measures like odds ratio and net reclassification improvement may not fully capture clinical utility.

Purpose of the Study:

• To introduce and review decision analysis as a method for evaluating the clinical value of new risk prediction markers.
• To highlight the limitations of purely accuracy-based measures in assessing marker utility.
• To propose the number needed to test (NNTest) as a decision-analytic measure for marker evaluation.

Main Methods:

• The study reviews the derivation and computation of the number needed to test (NNTest(T)) from risk stratification tables.
• NNTest(T) is defined as the minimum number of marker tests required per true positive for risk prediction to be worthwhile.
• The approach incorporates the ratio of anticipated harm from false positives to the benefit of true positives, establishing a risk threshold (T).

Main Results:

• Decision analysis, through NNTest(T), provides a measure of marker value that considers clinical implications beyond predictive accuracy.
• Comparison with accuracy measures in six studies demonstrated the advantages of the decision-analytic approach.
• The acceptability of NNTest(T) relative to test invasiveness and adverse consequences guides marker recommendation.

Conclusions:

• The number needed to test (NNTest) offers a clinically relevant framework for evaluating new risk prediction markers.
• This decision-analytic approach complements traditional accuracy measures by incorporating cost-benefit considerations.
• NNTest(T) facilitates informed decisions on incorporating new markers into clinical practice to optimize patient care.