Using treatment-tradeoff preferences to select diagnostic strategies: linking the ROC curve to threshold analysis

Selection of optimal diagnostic strategies depends on the accuracy of diagnostic tests, the prevalence of disease, and the relative benefits and harms resulting from test/treatment choices. One can characterize diagnostic strategies according to their net benefit-to-harm ratios. Within the framework of expected-utility theory, the benefit-to-harm ratio is equivalent to the marginal tradeoff between erroneous and correct treatment choices. Consequently, a physician can identify his or her preferred strategy by asking the question: "How many additional treatment errors am I willing to make in order to treat one additional person correctly?" Family physicians were asked to indicate their lowest and highest acceptable tradeoff ratios in a survey about streptococcal pharyngitis. All 53 respondents indicated uncertainty about their preferred treatment-tradeoff ratios. For 58% of respondents, the midpoints of the acceptable range of tradeoff ratios corresponded to generally accepted diagnostic strategies: 1) test all or 2) treat if classic, test all others. In contrast, previously published patient utilities yield a treatment-tradeoff ratio that corresponds to the strategy of treating all cases. The analysis illustrates the linkage between the optimal operating point on a receiver operating characteristic (ROC) curve and the selection of a preferred diagnostic strategy based on treatment thresholds derived from benefit-to-harm ratios. The survey results indicate that physicians can respond directly to questions assessing their preferences for such treatment thresholds. Differences between patients and physicians have significant impacts on choices of test and treatment strategies.