Multi-Domain Image Completion for Random Missing Input Data.

Area of Science:

• Computer Vision
• Medical Imaging
• Machine Learning

Background:

• Multi-domain data enhance vision applications, but missing data across modalities (e.g., multi-parametric MRI) hinders universal model development.
• Variations in data availability and imaging protocols present challenges for robust multi-domain analysis in real-world scenarios.

Purpose of the Study:

• To propose a general approach for completing randomly missing data across multiple domains in vision applications.
• To develop a novel multi-domain image completion method leveraging generative adversarial networks (GANs) with representational disentanglement.
• To demonstrate the utility of learned representations for downstream tasks like segmentation through a unified framework.

Main Methods:

• A generative adversarial network (GAN) with a representational disentanglement scheme was developed to extract shared content and separate style encodings.
• A unified framework integrating image completion and segmentation, utilizing a shared content encoder, was introduced.
• The method was evaluated on three distinct datasets for brain tumor segmentation, prostate segmentation, and facial expression completion.

Main Results:

• Consistent performance improvements were observed across all three tested datasets.
• The proposed method effectively handles randomly missing domain data, enhancing model generalizability.
• Learned representations from image completion proved beneficial for segmentation tasks.

Conclusions:

• The developed multi-domain image completion method effectively addresses missing data challenges in vision applications.
• The unified framework demonstrates the potential of leveraging image completion representations for high-level tasks like segmentation.
• This approach offers a promising solution for building more robust and versatile multi-domain models.