Constructing a Unified Vision-Language Model for Chest Radiograph-based Diagnostics, Medical Education, and Data...

Area of Science:

• Artificial Intelligence in Radiology
• Multimodal Large Language Models
• Medical Imaging Analysis

Background:

• Current AI tools often focus on single tasks within radiology.
• Integrating multiple AI functionalities can streamline radiologic workflows.
• The need for advanced AI for computer-aided diagnosis (CAD) and decision support is growing.

Purpose of the Study:

• To develop MedXChat, an LLM integrating radiology report generation, VQA, and text-to-image synthesis.
• To evaluate MedXChat's performance using computational metrics and expert radiologist assessments.
• To assess its potential for enhancing radiologic workflows and education.

Main Methods:

• Retrospective study training MedXChat on the MIMIC-CXR database (270,790 chest radiograph-report pairs, 54,138 VQA samples, 7500 text-to-image pairs).
• Computational evaluation using F1 score, AUC, and FID.
• Expert radiologist assessment of 50 random outputs for accuracy, consistency, and clinical alignment.

Main Results:

• MedXChat achieved superior AUC (0.67) and F1 score (0.44) in report generation compared to existing models.
• Higher accuracy in VQA for edema (73%) and pleural effusion (80%) compared to LLM-CXR and LLaVA-Med.
• Lowest FID (43.46) and highest classification accuracy (71.5%) in text-to-image synthesis, producing high-quality images.

Conclusions:

• MedXChat successfully integrated report generation, VQA, and image synthesis into a unified framework.
• Achieved state-of-the-art performance across all evaluated tasks.
• Shows promise for supporting professional applications, improving radiologic workflows, education, and data augmentation.