Comparison of Chest Radiograph Captions Based on Natural Language Processing vs Completed by Radiologists

Area of Science:

• Radiology
• Artificial Intelligence
• Medical Informatics

Background:

• Artificial intelligence (AI) shows potential in interpreting chest radiography (CXR) abnormalities and generating descriptive captions.
• A prospective evaluation is crucial to determine the practical utility of AI-driven captioning in clinical workflows.

Purpose of the Study:

• To prospectively compare the efficiency and accuracy of natural language processing (NLP)-generated CXR captions against traditional radiologist diagnostic findings.
• To assess the impact of NLP-generated captions on the radiologist reporting time and report consistency.

Main Methods:

• A multicenter diagnostic study utilized a large dataset of CXRs and reports for training AI models.
• A bidirectional encoder representation from transformers model extracted linguistic features to train convolutional neural networks for abnormality detection.
• Prospective testing involved residents drafting reports using either normal templates, rule-based captions, or NLP-generated captions, with final review by experienced radiologists.

Main Results:

• The NLP-generated caption model significantly reduced resident reporting time (283 seconds) compared to normal templates (347 seconds) and rule-based models (296 seconds).
• NLP-generated captions demonstrated the highest similarity to final radiologist reports (Bilingual Evaluation Understudy score of 0.69), outperforming normal templates (0.37) and rule-based models (0.57).
• The diagnostic performance, measured by the area under the curve for abnormal signs, was comparable across retrospective (0.87) and prospective (0.84) datasets.

Conclusions:

• Natural language processing (NLP)-generated CXR captions enhance radiologist efficiency by streamlining the reporting process.
• The use of NLP-generated captions maintains high consistency with expert radiologist findings, indicating their clinical relevance and reliability.
• This study validates the practical value of AI-powered captioning in improving diagnostic workflows for chest radiography.