Hierarchical Barycentric Multimodal Representation Learning for Medical Image Analysis.

Area of Science:

• Medical image analysis
• Machine learning
• Geometric deep learning

Background:

• Multimodal medical image analysis uses diverse data (e.g., MRI, DTI, PET) to improve diagnostics.
• Learning robust representations is crucial but challenging with missing data and varied distributions.
• Existing methods lack theoretical grounding in the geometric behavior of multimodal data.

Purpose of the Study:

• Introduce a generalized geometric framework for multimodal representation learning.
• Unify existing methods using a barycentric perspective.
• Propose a novel approach using Wasserstein barycenters with hierarchical priors.

Main Methods:

• Developed a generalized geometric perspective based on barycenters.
• Implemented generalized Wasserstein barycenters with hierarchical modality-specific priors.
• Evaluated the framework on brain tumor MRI segmentation and normative modeling tasks.

Main Results:

• Demonstrated consistent improvements over existing multimodal approaches.
• Showcased enhanced representation quality by preserving unimodal distribution geometry.
• Validated the framework's effectiveness on key clinical applications.

Conclusions:

• The proposed barycentric formulation offers a unified theoretical understanding.
• The novel approach effectively handles missing modalities and heterogeneous data.
• Scalable, theoretically grounded methods advance robust representation learning in medical imaging.