Identification of clinically important changes in health status using receiver operating characteristic curves

Identification of criterion standards for clinically important changes for groups of patients requires that judgments of the degree of change that represents a clinically important change are consistent among patients. We demonstrate the use of receiver operating characteristic (ROC) curves to test if patients' judgments of clinically important changes are consistent. Twenty-three patients with systemic lupus erythematosus (SLE) were examined prospectively every 2 weeks for up to 40 weeks. At each assessment, each patient rated the activity of their SLE on a visual analog scale, rated whether their SLE was more active, less active, or unchanged over each 2-week interval, and rated the importance of any change in SLE activity. One of three physician examiners completed similar assessments. Each measured change in the patient global assessment was categorized according to the patient's judgment of whether no change in SLE activity was noted or whether the patient thought their SLE was more or less active during the interval. ROC curves were constructed from these data. Areas under the ROC curve that were significantly greater than 0.5 were considered evidence for consistent ratings among patients of important changes in SLE activity. Patient assessments of change were available for 383 of 392 2-week intervals (97.7%). Of these, patients reported no change in SLE activity in 200 intervals, improvement in 72 intervals, and worsening in 111 intervals. Intervals of improvement could be distinguished from intervals of no change by changes in the patient global assessments [ROC area = 0.68; 95% confidence interval (CI) 0. 60, 0.76]. The cutpoint with the greatest sensitivity and specificity for any improvement was a decrease of 5 points or more (on a 0-100 scale) in patient global assessment. Intervals of worsening could also be distinguished from intervals of no change (ROC area = 0.80; 95% CI 0.74, 0.85), and the best cutpoint was an increase of 5 points or more in the patient global assessment. Group criteria for major improvement or worsening and for relative changes in the patient global assessment could also be determined, as could criteria for important changes in physician global assessments. By testing the consistency of patients' judgments of important changes, ROC curves provide a means to determine if group criteria for clinically important change can be established.