Detecting errors in a scoring program: a method of double diagnosis using a computer-generated sample

This paper discusses a new method for locating errors in diagnostic computer scoring programs for structured clinical interviews. It was proposed as a test of the accuracy of the scoring program for the Composite International Diagnostic Interview, version 1.1. The proposal was to create an independent scoring program in a different computer language but serving the same criteria. Both programs were then applied to the same large set of valid (i.e., logically consistent) computer-generated test cases, and differences in diagnostic assignments reviewed. The method described can identify the program steps that account for the sources of the errors. Corrections can be made and the programs run again on new sets of test cases until discrepancy-free results are achieved. While this method cannot discover errors that are repeated in the two programs, it does discover more of the errors in a scoring program than we have previously been able to identify. This technique provides a systematic and rigorous approach to assuring the accuracy of scoring programs based on established algorithms.