Vision-language model for report generation and outcome prediction in CT pulmonary angiogram

Area of Science:

• Artificial Intelligence in Medical Imaging
• Radiology and Diagnostic Imaging
• Computational Pathology

Background:

• Pulmonary embolism (PE) interpretation from CTPA scans is challenging for current AI.
• Existing AI tools lack specificity and structured reporting capabilities.
• Accurate PE diagnosis is critical for patient outcomes.

Purpose of the Study:

• To develop an agent-based AI framework for enhanced PE detection and reporting.
• To integrate Vision-Language Models (VLMs) and Large Language Models (LLMs) for comprehensive analysis.
• To improve structured reporting and outcome prediction in PE patients.

Main Methods:

• An agent-based framework combining VLMs and LLMs was developed.
• The model was trained on over 69,000 CTPA studies from multiple institutions.
• Abnormality classification, report generation, and survival prediction were evaluated.

Main Results:

• The framework achieved strong performance in classifying 32 PE-related abnormalities.
• Abnormality-guided reporting outperformed baseline methods.
• A multimodal fusion model improved survival prediction over traditional scores.

Conclusions:

• The proposed AI framework offers a clinically meaningful solution for PE diagnosis.
• It enables end-to-end interpretation, structured reporting, and outcome prediction.
• This approach advances AI applications in medical imaging for PE management.