Sensitivity and specificity of a screening questionnaire for dry eye

We developed a Dry Eye Screening Questionnaire for the Dry Eye Epidemiology Projects (DEEP), a proposed large epidemiologic study. All persons who screen positive and a small sample of those who screen negative are to be invited for a diagnostic examination. Containing 19 questions, of which only 14 were used in the analysis, the questionnaire takes only a few minutes to administer on the telephone. To construct a discriminator function and thus a ROC curve, we used stepwise multiple regression on screening responses from a clinic series of 77 cases and 79 controls. Stepwise regression may incorporate into the predictor equation variables whose relation to the predicted is only accidental. Further, misclassification rates are underestimated by the resubstitution method, in which the proportion misclassified is obtained from the same dataset in which the discriminator function was fitted. To counter these problems, we randomly divided the data in half. We chose as predictors only those variables (Dry and Irritated) selected by stepwise regression in both data halves. We estimated unbiased misclassification rates using the unbiased test set method, in which the discriminator is fitted in one data half, and misclassification rates are calculated in the other half. Comparison of ROC curves arising from resubstitution and test set estimates indicates that resubstitution bias in misclassification rate estimation is negligible in our data. A resubstitution estimate made on the entire data is thus preferred. The resulting sensitivity/specificity values are reasonably high (e.g., 60%/94%), suggesting that the questionnaire will be a useful screening tool in the DEEP study. A second discriminator using the sum of all 14 responses is similar in its misclassification characteristics to the first discriminator. A second potentially significant error, arising from applying results from a clinical series to a general population, will be investigated as survey results in DEEP become available.