[Use of expert systems in internal medicine. Analysis of performance and reliability of the Quick Medical Reference]

Here are reported the results of an analysis on the performance and the reliability of an expert system as a diagnostic aid in internal medicine, the Quick Medical Reference (QMR). The QMR is able to suggest a list of differential diagnoses elaborated on the basis of the clinical data available from patient's study cases. The study was performed on 27 clinicopathological cases published by the N Engl J Med in a consecutive series starting from 1993 until 1994. Each case was analyzed in three different phases in the attempt to simulate real life diagnostic procedure. In phase I only anamnestic and physical examination data were used, in phase II laboratory routine data, in phase III all the remaining clinical data. The QMR performance was evaluated by determining the frequency in which the "correct diagnosis" was given as highly probable by QMR in each of the three phases. At the end of phase III, the program suggested the correct diagnosis as the only one possible in 1 case. In 25 out of the 27 cases studied, the correct diagnosis was ranked within the first 5 positions, 14 out of 27 cases were listed as the most probable hypothesis. In 48% of the cases the program was able to give a diagnostic indication only on the basis of anamnestic and physical examination data. The program gave a poor performance only in those cases that were not well represented or the method used for the diagnostic procedure was not accurate.