HF-VLP: A multimodal vision-language pre-trained model for diagnosing heart failure.

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Natural Language Processing for Healthcare

Background:

• Rising heart failure (HF) incidence necessitates advanced diagnostic tools.
• Multimodal vision-language pretraining shows promise for medical diagnosis.
• Challenges include noisy labels in medical datasets and the need for efficient model adaptation.

Purpose of the Study:

• To develop a multimodal vision-language pretrained model (HF-VLP) for improved heart failure detection.
• To address challenges of noisy labels and enhance model efficiency for clinical application.
• To fuse chest X-ray and radiology report data for comprehensive patient assessment.

Main Methods:

• Developed HF-VLP, a multimodal vision-language pretrained model for heart failure.
• Implemented label calibration loss to mitigate noisy labels during pretraining.
• Utilized decomposed singular value weight-decomposed low-rank adaptation (PEFT) for efficient fine-tuning (<1% parameters).
• Integrated chest X-ray and radiology report features via a dynamic fusion graph module.

Main Results:

• Achieved average AUC of 83.67% on the Open-I dataset and 91.28% on the PPL-CXR dataset for multisymptom prediction.
• The parameter-efficient fine-tuning method improved diagnosis rates compared to zero-shot.
• The model demonstrated accurate classification of patient symptoms, aiding clinical diagnosis.

Conclusions:

• HF-VLP effectively integrates multimodal medical data for heart failure diagnosis.
• The developed methods address key challenges in medical pretraining, including noisy labels and efficient adaptation.
• This model shows significant potential to assist clinicians in the early and accurate diagnosis of heart failure.