Modality Bank: Learn multi-modality images across data centers without sharing medical data.

Area of Science:

• Medical image analysis
• Artificial intelligence in healthcare
• Computer vision

Background:

• Multi-modality images offer comprehensive data for medical analysis but are costly and difficult to acquire across institutions.
• Current limitations hinder the full utilization of multi-modality information in clinical settings.

Purpose of the Study:

• To propose ModalityBank, a privacy-secured, decentralized multi-modality adaptive learning architecture.
• To enable learning from comprehensive multi-modality data despite acquisition challenges.
• To facilitate modality completion and enhance medical image analysis.

Main Methods:

• Developed a decentralized adaptive learning architecture (ModalityBank) using domain-specific modulation parameters within a domain-agnostic network.
• Implemented a generator capable of producing high-quality images for specific modalities by switching configurations.
• Enabled modality completion across data centers by synthesizing missing image modalities.

Main Results:

• The proposed method successfully generates high-quality single-modality images.
• ModalityBank effectively completes missing modalities across decentralized data centers.
• Downstream tasks trained on synthesized multi-modality data outperformed those trained on single real data centers, achieving near real-data performance.

Conclusions:

• ModalityBank offers a privacy-secured and efficient solution for leveraging multi-modality medical image data.
• The architecture facilitates modality completion, overcoming acquisition limitations.
• Synthesized multi-modality data improves the performance of downstream medical image analysis tasks.