Diagnostic Accuracy and Clinical Value of a Domain-specific Multimodal Generative AI Model for Chest Radiograph...

Area of Science:

• Radiology
• Artificial Intelligence
• Medical Imaging

Background:

• Generative artificial intelligence (AI) is poised to transform radiology workflows.
• Clinical value assessment is crucial for AI in frequent examinations like chest radiograph interpretation.

Purpose of the Study:

• To develop and evaluate a domain-specific multimodal generative AI model for preliminary chest radiograph interpretations.
• Assess the diagnostic accuracy and clinical utility of this specialized AI model.

Main Methods:

• Retrospective collection of 8,838,719 radiograph-report pairs for training.
• Testing on 2,145 radiographs from public datasets and excluded training data.
• Evaluation of sensitivity, specificity, and subjective quality (acceptability, agreement, ranking) by four radiologists, comparing the domain-specific AI with radiologist reports and a general-purpose AI (GPT-4Vision).

Main Results:

• The domain-specific AI model achieved high sensitivity for critical findings: 95.3% for pneumothorax and 92.6% for subcutaneous emphysema.
• Radiologist acceptance rate for AI-generated reports was 70.5%, significantly higher than GPT-4Vision (29.6%).
• AI-generated reports received the highest agreement (median=4) and quality (median=4) scores, and were most frequently ranked highest by radiologists.

Conclusions:

• A domain-specific multimodal generative AI model demonstrates significant potential for high diagnostic accuracy.
• The AI model shows considerable clinical value in providing preliminary chest radiograph interpretations for radiologists.
• This specialized AI offers a promising tool to augment radiology workflows.