Automated medical knowledge acquisition: a study of consistency

Knowledge bases are more representative of the population of medical experts if they are constructed by a group of individuals, rather than one practitioner. However, one runs into problems with consistency when information is elicited from a group without a consistent format and terminology. This study examines the consistency of relatively unconstrained computer-elicited medical knowledge using the computer program, KSSO. The results of this study show that the group of ten general internists were somewhat consistent in the diagnoses they listed for a patient presenting with chest pain. They were much less consistent in the findings they listed to differentiate between the diagnoses they had listed. The mean number of subjects listing each diagnosis was 3.3 +/- 2.7 while the mean for findings was 2.0 +/- 1.5. The implications of these data are discussed.