Automated Ensemble Multimodal Machine Learning for Healthcare.

Area of Science:

• Artificial Intelligence in Medicine
• Machine Learning for Healthcare
• Multimodal Data Fusion

Background:

• Current machine learning models in healthcare often rely on single data types, unlike clinical practice which integrates multiple sources.
• Challenges persist in developing and adopting multimodal machine learning systems for clinical use.

Purpose of the Study:

• To introduce AutoPrognosis-M, a novel multimodal framework for integrating structured clinical data and medical imaging.
• To address the limitations of single-modality approaches in medical AI and facilitate clinical adoption.

Main Methods:

• Developed AutoPrognosis-M, an automated machine learning framework.
• Integrated 17 imaging models (CNNs, Vision Transformers) and structured clinical data.
• Employed three distinct multimodal fusion strategies, utilizing ensemble learning.

Main Results:

• Demonstrated the importance of multimodal machine learning through an application on a skin lesion dataset.
• Highlighted the effectiveness of combining multiple fusion strategies via ensemble learning.

Conclusions:

• AutoPrognosis-M facilitates the integration of diverse data types for improved medical predictions.
• The open-sourced framework aims to accelerate multimodal machine learning adoption and innovation in healthcare.