Retrieval-based adaptive fusion strategy for medical report generation

Area of Science:

• Medical imaging and natural language processing.
• Artificial intelligence in healthcare.
• Clinical informatics.

Background:

• Existing retrieval-based medical report generation methods struggle with redundancy and misdescription, particularly for high-resolution images.
• Current evaluation metrics for report generation often overlook clinical implications of misdescriptions.
• A high-resolution CT-report dataset with 9 categories and 505 patients was created.

Purpose of the Study:

• To develop an improved retrieval-based medical report generation strategy.
• To enhance the accuracy and clinical interpretability of generated radiology reports.
• To introduce a more robust evaluation metric for medical report generation.

Main Methods:

• Proposed RAFS (Retrieval-based Adaptive Fusion Strategy) to dynamically balance retrieval and generation modules.
• Integrated an attention mechanism for calculating word similarity and determining retrieval probabilities.
• Developed DICE, a dual-perspective evaluation metric incorporating positive scoring and misdescription penalties.

Main Results:

• RAFS achieved superior performance across standard metrics (BLEU-4, METEOR, ROUGE_L, CIDEr) in CT report generation.
• The proposed DICE score averaged 64.6, indicating improved clinical evaluation.
• RAFS outperformed existing methods in generating accurate and clinically relevant reports.

Conclusions:

• RAFS significantly enhances the clinical interpretability of generated medical reports.
• The study highlights the importance of adaptive fusion strategies and comprehensive evaluation metrics.
• Future work will focus on refining the characterization of local pathological lesions in generated reports.