Multi-Label Generalized Zero Shot Chest X-Ray Classification by Combining Image-Text Information With Feature...

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Computer Vision

Background:

• Medical image classification models require diverse data for robustness.
• Generalized Zero Shot Learning (GZSL) enables prediction of seen and unseen classes.
• Existing GZSL methods often overlook the multi-label nature of chest X-rays.

Purpose of the Study:

• To develop a novel multi-modal multi-label GZSL approach for chest X-ray classification.
• To address the challenge of classifying multiple diseases in unseen X-ray images.
• To enhance the robustness and accuracy of medical image classification models.

Main Methods:

• Leveraging feature disentanglement and multi-modal information for feature synthesis.
• Utilizing BioBert for text embeddings of disease labels and constructing a label similarity dictionary.
• Employing graph aggregation and clustering to learn inter-label similarities and identify representative vectors.
• Synthesizing multi-label disease samples for both seen and unseen classes.

Main Results:

• The proposed method successfully generates realistic multi-label disease samples.
• Demonstrated superior performance compared to existing methods on NIH and CheXpert datasets.
• Effectively synthesizes features for unseen classes in a multi-label context.

Conclusions:

• The novel multi-modal multi-label GZSL approach significantly advances chest X-ray classification.
• Feature disentanglement and multi-modal information are crucial for handling unseen disease classes.
• The method offers a promising direction for robust medical image analysis with multiple labels.