Cohesive Multi-Modality Feature Learning and Fusion for COVID-19 Patient Severity Prediction.

Area of Science:

• Medical Informatics
• Artificial Intelligence
• Radiology

Background:

• The COVID-19 pandemic overwhelmed healthcare systems globally in 2020.
• Accurate patient severity assessment is crucial for resource allocation and infection control.
• Traditional diagnostic methods face challenges during widespread outbreaks.

Purpose of the Study:

• To develop an end-to-end model for predicting COVID-19 patient severity.
• To integrate multi-modality data, including electronic medical records (EMR) and chest CT scans.
• To improve the robustness and performance of severity prediction models.

Main Methods:

• Proposed a multi-modality feature learning and fusion model.
• Utilized High-order Factorization Network (HoFN) for clinical feature analysis from EMR.
• Employed an attention-based deep convolutional neural network (CNN) for chest CT image analysis.
• Designed a cross-modality feature fusion loss function.

Main Results:

• The model achieved high performance in predicting COVID-19 patient severity in a real-world setting.
• The fusion approach demonstrated improved accuracy and robustness, even with missing data modalities.
• HoFN effectively learned clinical feature interactions without manual engineering.
• CNN accurately processed lung CT images for severity indicators.

Conclusions:

• The proposed multi-modality model offers an effective solution for COVID-19 patient severity assessment.
• This AI-driven approach can aid in better hospital management and resource distribution during pandemics.
• The model's ability to handle missing data enhances its practical applicability.