Scalable Unimodal and Multimodal Deep Learning for Multi-Label Chest Disease Detection: A Comparative Analysis

Area of Science:

• Artificial Intelligence in Medicine
• Medical Imaging Analysis
• Computational Pathology

Background:

• Accurate chest disease diagnosis is challenging, especially with coexisting pathologies.
• Current deep learning models often use unimodal data, limiting clinical applicability.
• This study addresses limitations by comparing unimodal and multimodal deep learning for chest disease classification.

Purpose of the Study:

• To compare unimodal and multimodal deep learning models for multi-label chest disease classification.
• To evaluate the impact of dataset scale on model performance and generalizability.
• To assess the effectiveness of integrating chest X-ray images with clinical metadata.

Main Methods:

• Developed twelve deep learning models using ResNet50, EfficientNetB3, and DenseNet121 architectures.
• Configured models for unimodal (image-only) and multimodal (image + clinical data) learning.
• Evaluated models on two NIH Chest X-ray Dataset scales (5,606 and 121,120 samples) using AUROC metrics.

Main Results:

• Multimodal models consistently outperformed unimodal models across all architectures and dataset sizes.
• Performance improvements were more significant with larger datasets.
• Increased data volume enhanced model generalization and reduced performance variance, especially for rare diseases.

Conclusions:

• Multimodal deep learning effectively enhances diagnostic accuracy for chest diseases.
• Integrating clinical data with medical images improves multi-label classification.
• Findings support developing robust clinical decision support systems for chest disease assessment.