Observe, align, and enhance: a hierarchical retrieval-augmented vision-language model for generating radiology...

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Automated radiology report generation (RRG) aids clinical diagnosis but struggles with aligning visual data to detailed narratives.
• Existing generative models face challenges in precise visual-to-text alignment for comprehensive diagnostic reports.

Purpose of the Study:

• To introduce a novel hierarchical retrieval-enhanced framework, Observe, Align, and Enhance (OAE), for improved radiology report generation.
• To address limitations in current RRG methods concerning semantic consistency and diagnostic precision.

Main Methods:

• The Observe phase uses retrieval to identify similar images and reports for enhanced visual feature comprehension.
• The Align phase generates an initial report guided by retrieved reports for semantic consistency.
• The Enhance phase iteratively refines the report with additional text for improved coherence and precision.

Main Results:

• The OAE framework demonstrated superior performance compared to state-of-the-art approaches on the IU-XRAY and MIMIC-CXR datasets.
• Evaluations showed improvements in both diagnostic accuracy and the overall quality of generated radiology reports.

Conclusions:

• The proposed OAE framework offers a more effective approach to automated radiology report generation.
• This retrieval-enhanced method enhances semantic coherence and diagnostic precision in clinical narratives.