Using Recurrent Neural Networks to Extract High-Quality Information From Lung Cancer Screening Computerized...

Area of Science:

• Radiology and Medical Informatics
• Natural Language Processing in Healthcare
• Artificial Intelligence in Medical Imaging

Background:

• Lung cancer screening programs generate numerous low-dose computed tomography (LDCT) reports.
• These reports contain valuable clinical information in free-text format.
• Automated information extraction is needed for quality control and analysis.

Purpose of the Study:

• To develop and evaluate high-performance named-entity recognition (NER) models for extracting clinically relevant information from LDCT reports.
• To compare the performance of different NER models, including rule-based and neural network approaches.
• To assess the utility of the best-performing model for inter-radiologist quality control.

Main Methods:

• Trained a rule-based model and two bidirectional long short-term memory (Bi-LSTM) NER neural network models.
• Utilized data from a longitudinal lung cancer screening program (8,305 reports).
• Compared model performance using F1 scores against a benchmark radiology NER model (Stanza) on an independent evaluation set.

Main Results:

• The Bi-LSTM NER recurrent neural network model achieved the highest overall F1 score (0.950), outperforming Stanza (0.872) and the rule-based model (0.809).
• High recall (0.916-0.991) and precision (0.892-0.997) were observed for the best model across different entity types.
• The model demonstrated stable performance over time and identified significant differences in reporting detail among radiologists.

Conclusions:

• An open-source Bi-LSTM NER model was developed that surpasses existing models in accuracy for extracting information from lung cancer screening CT reports.
• This model enables efficient auditing and feedback mechanisms.
• The tool has the potential to significantly improve the quality of patient care through enhanced quality control.