Test-retest reliabilities of two treatment-preference instruments in measuring utilities

The authors assessed the test-retest reliabilities of two treatment-preference instruments recently applied to the measurement of the utilities of health states after different treatment modalities for cancer. The first instrument measures the strengths of preferences concerning a choice between a wait-and-see policy, and treatment with radiotherapy after an initial surgical breast-conserving procedure for early breast cancer. The second measures the strengths of preferences concerning a choice between two hypothetical surgical treatment outcomes in cancer of the rectum with different probabilities of expected five-year survival. Both measure the strength of a subject's treatment preference given probabilities of treatment-related costs and benefits. The subjects were radiotherapy technicians (n = 20) and cancer patients (n = 20) who were interviewed in weeks 2 and 4 of radiotherapy. The test-retest reliabilities of both instruments were inconsistent and moderately high, with Spearman's rank correlations ranging from 0.38 to 0.81 and weighted kappas ranging from 0.38 to 0.69. To investigate whether the start of treatment with radiotherapy influenced the utilities that patients assigned to health states, the same procedure was applied in another, comparable, group of patients with cancer (n = 20). For this group, the first assessment was made prior to the start of treatment and the second during the second week of radiation therapy. The scores of this group of patients indeed appeared to be less stable than the scores of the patients assessed in weeks 2 and 4 of radiotherapy. However, the instability of the scores could have been the result of test bias.