Development of automated detection of radiology reports citing adrenal findings

The aim of this study was to determine the feasibility of automated detection of adrenal nodules, a common finding on CT, using a newly developed search engine that mines dictated radiology reports. To ensure Health Insurance Portability and Accountability Act compliance, we utilized a preexisting de-identified database of 32,974 CT reports from February 1, 2009 to February 28, 2010. Common adrenal descriptors from 29 staff radiologists were used to develop an automated rule-based algorithm targeting adrenal findings. Each sentence within the free text of reports was searched with an adapted NegEx negation algorithm. The algorithm was refined using a 2-week test period of reports and subsequently validated using a 6-week period. Manual review of the 3,693 CT reports in the validation period identified 222 positive reports while the algorithm detected 238 positive reports. The algorithm identified one true positive report missed on manual review for a total of 223 true positive reports. This resulted in a precision of 91% (217 of 238) and a recall of 97% (217 of 223). The sensitivity of the query was 97.3% (95% confidence interval (CI), 93.9-98.9%), and the specificity was 99.3% (95% CI, 99.1-99.6%). The positive predictive value of the algorithm was 91.0% (95% CI, 86.6-94.3%), and the negative predictive value was 99.8% (95% CI, 99.6-99.9%). The prevalence of true positive adrenal findings identified by the query (7.1%) was nearly identical to the true prevalence (7.2%). Automated detection of language describing common findings in imaging reports, such as adrenal nodules on CT, is feasible.