Leveraging an Image-Enhanced Cross-Modal Fusion Network for Radiology Report Generation

Area of Science:

• Artificial Intelligence in Medical Imaging
• Natural Language Processing for Healthcare
• Radiology and Diagnostic Imaging

Background:

• Automated radiology report generation (RRG) aims to assist radiologists, improve diagnostic accuracy, and optimize resource allocation.
• Existing RRG methods often struggle with low-quality images, lack of cross-modal information integration, and high latency.
• There is a need for advanced models that can enhance feature extraction from suboptimal medical images and efficiently generate reports.

Purpose of the Study:

• To develop an advanced model, the Image-Enhanced Cross-Modal Fusion Network (IFNet), for automatic radiology report generation.
• To address limitations in current RRG by enhancing feature extraction from low-quality images and incorporating cross-modal interactions.
• To improve the efficiency and suitability of RRG models for low-resource environments.

Main Methods:

• Proposed the Image-Enhanced Cross-Modal Fusion Network (IFNet) comprising three key modules.
• An image enhancement module to improve the representation of structures in X-ray images.
• Cross-modal fusion networks to capture interactions between image and text features.
• An efficient transformer-based module for optimized report generation, suitable for low-resource devices.

Main Results:

• IFNet demonstrated significant improvements in radiology report generation compared to existing state-of-the-art methods.
• The image enhancement module successfully boosted the detection rates by improving the detailed representation of image structures.
• Experimental results on the IU X-ray and MIMIC-CXR datasets validated the effectiveness of IFNet.

Conclusions:

• The proposed IFNet effectively addresses key challenges in automatic RRG, including low-quality image analysis and efficient report generation.
• IFNet offers a promising solution for enhancing the capabilities of computer-aided diagnostic tools in radiology.
• The model's efficiency makes it suitable for deployment in resource-constrained healthcare settings.