Assessing diagnostic tests once an optimal cutoff point has been selected

Area of Science:

• Biostatistics
• Diagnostic Test Evaluation
• Medical Informatics

Background:

• The accuracy of quantitative diagnostic tests relies on the selected cutoff point.
• Commonly, cutoff points are chosen to maximize the sum of specificity and sensitivity based on observed data.

Purpose of the Study:

• To investigate the impact of selecting cutoff points that maximize specificity plus sensitivity.
• To evaluate the bias introduced by this common practice, particularly in small sample sizes.
• To introduce and apply the bootstrap procedure for bias correction and standard error estimation.

Main Methods:

• Simulation studies were used to assess test performance with different cutoff selection methods.
• The bootstrap procedure, a Monte Carlo statistical method, was applied.
• Independent and paired comparisons of diagnostic tests were analyzed.

Main Results:

• The practice of maximizing specificity plus sensitivity leads to optimistic performance assessments, with bias up to 15% for small sample sizes (n=25).
• Binomial standard errors for specificity and sensitivity estimates are inaccurate under this selection method.
• The bootstrap procedure effectively corrects for bias and provides reliable standard error estimates, including for the optimal cutoff point.

Conclusions:

• The common method for selecting diagnostic test cutoff points introduces optimism bias, especially in small sample sizes.
• The bootstrap procedure is a robust statistical method for correcting bias and estimating standard errors in diagnostic test evaluation.
• Binomial tests are satisfactory for comparing diagnostic tests when optimal cutoff points are selected.