Evaluating Risk Prediction with Data Collection Costs: Novel Estimation of Test Tradeoff Curves.

Area of Science:

• Biostatistics
• Medical Decision Making
• Health Informatics

Background:

• The test tradeoff curve aids in evaluating the utility of risk prediction data for treatment decisions.
• It defines the minimum data collection points per true positive to justify risk prediction based on benefit-cost ratios or risk thresholds.
• A high test tradeoff, like 3,000 tests per true positive cancer prediction, may indicate that risk prediction is not cost-effective.

Purpose of the Study:

• To introduce a novel method for estimating the test tradeoff curve using individual risk scores.
• To provide a more straightforward and appealing alternative to previous methods that required grouping risk scores.
• To evaluate the performance of the new method using synthetic datasets.

Main Methods:

• Estimates a concave receiver-operating characteristic (ROC) curve using individual risk scores.
• Constructs a concave envelope of ROC points.
• Employs a slope-based moving average and minimizes the sum of squared errors.
• Connects successive ROC points with line segments to derive the test tradeoff curve.

Main Results:

• The new method successfully estimates a concave ROC curve from individual risk scores.
• The estimated concave ROC curve yields a corresponding estimated test tradeoff curve.
• Analyses of two synthetic datasets demonstrate the method's applicability and effectiveness.

Conclusions:

• Estimating the test tradeoff curve with individual risk scores is simple to implement.
• This approach is more advantageous than prior methods necessitating risk score grouping.
• The method provides a valuable tool for optimizing data collection in risk prediction for clinical decision-making.