MARIA: A multimodal transformer model for incomplete healthcare data

Area of Science:

• Artificial Intelligence in Healthcare
• Deep Learning for Medical Data Analysis
• Multimodal Data Integration

Background:

• Integrating multimodal data is crucial for advanced healthcare models.
• Missing data presents a significant obstacle in real-world medical applications.
• Current methods often rely on imputation, which can introduce bias.

Purpose of the Study:

• To introduce MARIA (Multimodal Attention Resilient to Incomplete datA), a novel deep learning model.
• To address the challenge of missing data in multimodal healthcare datasets.
• To enhance the robustness and accuracy of diagnostic and predictive models.

Main Methods:

• MARIA employs a transformer-based architecture with an intermediate fusion strategy.
• It utilizes a modified masked self-attention mechanism to process only available data.
• The model avoids data imputation, preventing the generation of synthetic values.

Main Results:

• MARIA demonstrated superior performance across 8 diagnostic and prognostic tasks compared to 10 state-of-the-art models.
• The model showed significant resilience to varying degrees of data incompleteness.
• It effectively minimized biases often associated with imputation methods.

Conclusions:

• MARIA offers a robust solution for handling incomplete multimodal healthcare data.
• Its performance and resilience make it suitable for critical healthcare applications.
• The open-source availability of MARIA encourages further research and development.