Hyperfusion: A hypernetwork approach to multimodal integration of tabular and medical imaging data for predictive...

Area of Science:

• Medical Imaging Analysis
• Machine Learning in Healthcare
• Data Fusion Techniques

Background:

• Integrating diverse clinical data like medical imaging and Electronic Health Records (EHRs) is vital for comprehensive patient assessment.
• Deep Neural Networks (DNNs) show promise in multimodal medical tasks, but effectively merging imaging with tabular data (clinical, demographic, genetic) is challenging.
• Current research actively seeks advanced methods for multimodal data fusion in clinical applications.

Purpose of the Study:

• To develop a novel framework for fusing clinical imaging and tabular data using hypernetworks.
• To condition image processing on Electronic Health Record (EHR) values for enhanced multimodal analysis.
• To improve the accuracy of medical applications by leveraging complementary information from imaging and EHR data.

Main Methods:

• A novel hypernetwork-based framework was developed to fuse medical imaging and tabular data.
• The framework conditions image processing using patient-specific EHR values and measurements.
• The approach was evaluated on two brain Magnetic Resonance Imaging (MRI) tasks: brain age prediction and Alzheimer's Disease (AD) classification.

Main Results:

• The proposed hypernetwork framework successfully integrated brain MRI and EHR data.
• Performance evaluation showed the framework outperformed single-modality models.
• The method achieved superior results compared to existing state-of-the-art MRI-tabular data fusion techniques.

Conclusions:

• The novel hypernetwork framework offers an effective solution for fusing medical imaging and tabular data.
• This approach enhances the accuracy of medical analyses, particularly in brain MRI applications.
• The method demonstrates significant potential for improving diagnostic and treatment decisions in healthcare.