Comparative study of human expertise and an expert system: application to the diagnosis of child's meningitis

We present a general method of statistical evaluation for expert systems, applied to a system for the diagnosis of child's meningitis. Fifty difficult clinical cases of child's meningitis were submitted to the system, to seven senior specialists and to seven young physicians. Multidimensional analysis of the diagnosis of the infection category reveals that the two groups of physicians separate naturally and that the system is located among the group of experts. The study of the agreement of the microbiological diagnosis and therapeutic advice shows that the advice of the two groups of physicians is significantly different and that the advice of the system is significantly closer to that of the experts. This result is confirmed by the study of therapeutic errors. This type of study allows one to classify the performance of the system among physicians having different levels of expertise without referring to an objective solution.