Multimodal representation learning for medical analytics - a systematic literature review.

Area of Science:

• Medical Informatics
• Machine Learning
• Data Science

Background:

• Machine learning analytics on single-type (uni-modal) medical data are established.
• Patient data is diverse, necessitating multimodal approaches for personalized care.
• Multimodal representation learning (MRL) creates shared latent spaces to enhance analytics.

Purpose of the Study:

• To review the landscape of MRL in medical applications.
• To identify opportunities for advancing medical analytics using MRL.
• To clarify how modalities are applied in MRL for medical tasks.

Main Methods:

• A framework was developed to position MRL techniques and medical modalities.
• Over 1000 papers on medical analytics were reviewed, classified, and positioned.
• An online tool was created for researchers and developers.

Main Results:

• MRL applications in medical informatics are currently sparse.
• Most MRL work focuses on diagnostic rather than prognostic tasks.
• Numerous potential modality combinations remain unexplored or under-explored.

Conclusions:

• Significant potential exists for MRL in unexplored medical task and modality combinations.
• This review guides researchers to identify under-explored areas and novel MRL techniques.
• Further research into diverse MRL applications can advance personalized medical care.