MvKeTR: Chest CT Report Generation With Multi-View Perception and Knowledge Enhancement

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Radiology

Background:

• Automated CT report generation (CTRG) aims to reduce clinician workload and enhance patient care.
• Existing CTRG methods struggle to integrate multi-view anatomical information and clinical expertise, limiting diagnostic accuracy.
• Accurate and reliable diagnosis from 3D CT volumes requires synthesizing information across views and incorporating domain knowledge.

Purpose of the Study:

• To develop a novel Multi-view perception Knowledge-enhanced TansfoRmer (MvKeTR) that mimics the clinical diagnostic workflow.
• To improve the integration of diagnostic information from multiple anatomical views and incorporate essential clinical expertise.
• To enhance the accuracy and reliability of automated CT report generation.

Main Methods:

• A Multi-View Perception Aggregator (MVPA) with view-aware attention synthesizes information from multiple anatomical views.
• A Cross-Modal Knowledge Enhancer (CMKE) retrieves similar reports to integrate domain knowledge.
• Kolmogorov-Arnold Networks (KANs) are used as fundamental building blocks for parameter efficiency and improved high-frequency component capture.

Main Results:

• The proposed MvKeTR method significantly outperforms prior state-of-the-art models on the CTRG-Chest-548K dataset across multiple metrics.
• Experiments demonstrate the effectiveness of integrating multi-view information and clinical knowledge for improved CTRG.
• The use of KANs contributed to better performance and reduced overfitting in the model.

Conclusions:

• The MvKeTR model represents a significant advancement in automated CT report generation by effectively mimicking clinical diagnostic processes.
• Integrating multi-view perception and knowledge enhancement leads to more accurate and reliable diagnostic reports.
• The proposed approach offers a promising direction for improving AI-assisted radiological diagnosis and clinical decision-making.