Improvement in user performance following development and routine use of an expert system

Hospital-acquired (nosocomial) infections represent a significant cause of prolonged inpatient days and additional hospital charges. In many hospitals, infection control nurses manually review positive microbiology culture results to monitor the incidence and prevalence of potential nosocomial infections. We have developed an expert system called GermWatcherTM, which uses the United States Centers for Disease Control and Prevention National Nosocomial Infection Surveillance criteria to classify microbiology results as potential nosocomial infections. In February 1993, we deployed GermWatcher at a large tertiary-care teaching hospital. In July 1993, we implemented a revised version of GermWatcher. With each version, we performed an evaluation of the program by comparing its electronic classification of positive culture results to the paper-based manual classification performed by three infection control nurses and one Infectious Disease specialist (gold standard). In the present study, we focus not on changes in the performance of the expert system, but on changes in performance among the infection control nurses. We found significant improvement in agreement and accuracy in the manual classification of cultures by the infection control nurses in the second evaluation compared to the first evaluation. We attribute this improved manual performance to the development of the expert system's rule base throughout the two evaluation phases and to the use of the expert system in the nurses' daily activities.