R2GenCSR: Mining Contextual and Residual Information for LLMs-based Radiology Report Generation.

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Natural Language Processing

Background:

• Large Language Models (LLMs) show promise in radiology report generation.
• Current methods use Transformers for visual feature extraction, leading to high computational costs.
• Extracting effective visual information for LLMs remains a challenge.

Purpose of the Study:

• To develop a novel, context-guided, and efficient framework for radiology report generation.
• To address the limitations of high computational complexity and suboptimal feature extraction in existing methods.

Main Methods:

• Introduced Mamba as a vision backbone with linear complexity, achieving performance comparable to Transformers.
• Implemented context retrieval using positively and negatively related samples during training to enhance feature representation.
• Integrated vision tokens, context information, and prompt statements to guide LLM-based report generation.

Main Results:

• The proposed framework demonstrates effectiveness in generating high-quality medical reports.
• Experiments on IU X-Ray, MIMIC-CXR, and CheXpert Plus datasets validate the model's performance.
• Mamba backbone provides comparable performance to Transformers with significantly reduced computational complexity.

Conclusions:

• The context-guided efficient radiology report generation framework effectively enhances LLM performance.
• The use of Mamba as a vision backbone offers an efficient alternative to Transformers.
• The method successfully generates high-quality radiology reports, validated across multiple datasets.