Domain Adaptation Meets Zero-Shot Learning: An Annotation-Efficient Approach to Multi-Modality Medical Image...

Area of Science:

• Medical Imaging
• Computer Vision
• Machine Learning

Background:

• Deep learning models struggle with generalization in medical imaging due to limited annotated data.
• Existing zero-shot learning (ZSL) methods, primarily for natural images, are unsuitable for medical imaging due to domain-specific terminology.

Purpose of the Study:

• To propose a new paradigm for ZSL in medical images using cross-modality information.
• To address the challenge of recognizing unseen medical conditions without prior examples.

Main Methods:

• Extracting relation prototypes (prior knowledge) from a pre-existing model.
• Developing a cross-modality adaptation module to transfer prototypes to the zero-shot model.
• Implementing a relation prototype awareness module and an inheritance attention module to enhance prototype utilization.

Main Results:

• The proposed framework was evaluated on cardiac and abdominal cross-modality datasets.
• Demonstrated significant performance improvement over existing state-of-the-art methods.
• Successfully enabled deep models to generalize to unseen medical image classes.

Conclusions:

• The novel ZSL paradigm effectively utilizes cross-modality information for medical image analysis.
• The proposed modules enhance the deep model's ability to inherit and utilize prior knowledge.
• This approach offers a promising solution for improving deep learning generalization in the medical domain.