Radiology image interpretation system: modified observer performance study of an image interpretation expert system

Application of computer-based expert systems to diagnostic medical problems has been described in many areas including clinical diagnosis and radiology. Expert systems are computer programs that contain encoded expert knowledge to provide expert advice. A modified observer-performance study was done comparing the efficacy of the Radiology Image Interpretation System (RIIS), an expert system that diagnoses focal bone abnormalities, and radiology residents on a known set of 44 abnormal and 10 normal cases. Modified receiver operating characteristic curves for four inexperienced residents, five experienced residents, and RIIS were generated using the set of known radiographs. The true-positive rates of RIIS and the residents at false-positive rates of 0.05, 0.15, and 0.20 were estimated using the modified receiver operating characteristics curve and were compared using a paired t test. On the average, the RIIS system was less accurate when compared with experienced and inexperienced residents but the difference was only significant for experienced residents at a false-positive rate of 0.05. RIIS performed better than inexperienced residents when RIIS was used by experienced residents but this difference was not significant.