Monitoring free-text data using medical language processing

In this paper, we describe a software system for automated monitoring of free-text data in a medical information system that we call RadTRAC (Radiology Text Report Analyzer and Classifier). RadTRAC uses a medical language processing tool and rules derived from statistical analysis of a database to process free-text chest X-ray (CXR) reports and identify reports that describe new or expanding neoplasms for the purpose of monitoring the follow-up of these patients. To evaluate the RadTRAC system, we examined a set of 470 consecutive radiology reports at the Veterans Administration Medical Center, Palo Alto, CA. We compared RadTRAC classification of CXR reports with retrospective expert classification of the reports and with clinical classification from CXR films as recorded in a logbook while the films were being read. The RadTRAC system had a sensitivity of 90% and a specificity of 82% using the logbook as the gold standard. This was similar to the performance of expert radiologists (sensitivity, 92%; specificity, 90%). We then reviewed the charts, appointment schedule, and subsequent X-ray reports of cases either in the logbook or that were identified by RadTRAC as needing follow-up. Two cases in the logbook could have potentially benefited from an automatic monitoring system to ensure follow-up. RadTRAC identified six confirmed new tumors or new metastatic lesions that were not in the logbook. Six other cases were identified by the RadTRAC system with suspicious X-ray findings that had either no follow-up or no further mention of the X-ray lesion in medical records. This suggests that a reminder system based on the RadTRAC technology would be potentially useful.