Weakly guided attention model with hierarchical interaction for brain CT report generation.

Area of Science:

• Medical Imaging Analysis
• Artificial Intelligence in Healthcare
• Natural Language Processing for Radiology

Background:

• Brain Computed Tomography (CT) report generation aids in diagnosing cerebrovascular diseases.
• Current methods struggle with aligning multiple images and sentences, and focusing on critical details.
• Limitations exist in accurately describing findings due to challenges in feature representation.

Purpose of the Study:

• To improve the accuracy and efficiency of Brain CT report generation.
• To address the many-to-many alignment challenge between multi-images and multi-sentences.
• To enhance the model's ability to attend to critical images and lesion areas.

Main Methods:

• Proposed a novel Weakly Guided Attention Model with Hierarchical Interaction (WGAM-HI).
• Employed a hierarchical interaction framework with a two-layer attention model and report generator.
• Introduced two weakly guided mechanisms using pathological events and Gradient-weighted Class Activation Mapping (Grad-CAM).

Main Results:

• WGAM-HI effectively performs many-to-many matching between visual images and semantic sentences.
• The model demonstrates improved focus on important images and lesion areas.
• Experimental results show more accurate report generation compared to existing methods.

Conclusions:

• WGAM-HI significantly enhances Brain CT report generation accuracy.
• The hierarchical interaction and weakly guided attention mechanisms are effective.
• This model offers a promising approach for automated radiological report generation.