VTAG: Visual-Textual Association Guided Radiology Reports Generation.

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Natural Language Generation

Background:

• Automated radiology report generation is vital for clinical efficiency and diagnostic accuracy.
• Current models struggle with interpretability and description accuracy.
• Need for advanced methods to bridge the gap between medical images and textual reports.

Purpose of the Study:

• To develop an integrated framework to enhance radiology report generation.
• To improve the accuracy and interpretability of automatically generated diagnostic reports.
• To address limitations of existing models in capturing comprehensive image information.

Main Methods:

• Proposed an integrated framework combining target detection and contextual alignment.
• Implemented a full-spectrum feature fusion method integrating high- and low-frequency image features.
• Validated the approach on the MIMIC-CXR public dataset.

Main Results:

• The VTAG method demonstrated superior performance over existing approaches on multiple metrics.
• Achieved a 14.3% improvement in the average of six traditional metrics compared to the MLRG model.
• The integrated framework enhanced the comprehensive and hierarchical understanding of medical images.

Conclusions:

• The VTAG framework effectively improves radiology report generation accuracy and interpretability.
• The combination of target detection, contextual alignment, and feature fusion offers a robust solution.
• This advancement holds significant potential for improving clinical workflows and diagnostic outcomes.