Bigram frequency analysis for detection of radiology report errors.

Area of Science:

• Radiology Informatics
• Natural Language Processing
• Medical Error Detection

Background:

• Radiology reports are crucial for patient care and require high accuracy.
• Identifying and rectifying errors in radiology reports is essential for patient safety.
• Automated methods for error detection can improve the efficiency and reliability of report review.

Purpose of the Study:

• To evaluate the effectiveness of bigram frequency analysis for detecting errors in radiology reports.
• To assess the utility of comparing observed bigram frequencies against expected frequencies derived from a large corpus.

Main Methods:

• A large corpus of 48,050 CT reports was used to establish baseline bigram frequencies.
• A test set of 400 reports (200 attending, 200 trainee) was analyzed.
• Bigrams in test reports with rare or absent frequencies in the corpus were flagged for manual review.

Main Results:

• The analysis flagged 682 n-grams in attending reports (11.6% true errors) and 1378 in trainee reports (7.9% true errors).
• The most common flagged bigrams were those not present in the corpus but composed of common words.
• Sensitivity for error detection was 58% in attending reports and 97% in trainee reports.

Conclusions:

• Bigram frequency analysis shows promise as a practical tool for identifying potential errors in radiology reports.
• Further refinement is needed to enhance the positive predictive value of this automated error detection method.