M3T-CKD: A multi-modal multi-teacher contrastive knowledge distillation framework for survival prediction of patients
Nana Jia1, Tong Jia2, Zhiao Zhang1
1College of Information Science and Engineering, Northeastern University, Shenyang 110819, Liaoning, China.
Insights
A new framework, M³T-CKD, effectively fuses ultrasound images and clinical data to predict survival in carotid atherosclerosis patients. This method improves cardiovascular disease risk assessment and patient outcomes.
Area of Science:
- Biomedical Engineering
- Medical Imaging Analysis
- Cardiovascular Disease Research
Background:
- Carotid atherosclerosis poses significant cardiovascular disease risk, necessitating improved patient survival prediction.
- Current survival prediction methods face challenges in effectively fusing multi-modal data (ultrasound images, clinical data) and accurate plaque segmentation.
- Accurate plaque characterization from ultrasound is crucial for predicting patient outcomes.
Purpose of the Study:
- To develop an advanced framework for enhanced survival prediction in patients with carotid atherosclerosis.
- To address the challenges of multi-modal data fusion and accurate plaque segmentation in ultrasound images.
- To leverage knowledge distillation for improved feature learning in survival prediction.
Main Methods:
- Proposed a multi-modal multi-teacher contrastive knowledge distillation (M³T-CKD) framework.
- Introduced a modality feature disentanglement (MFD) module for fusing shared and specific features from ultrasound and clinical data.
- Implemented a spatial-channel decoupling learning scheme for improved plaque segmentation and a contrastive knowledge distillation (CKD) module for task-specific feature learning.
Main Results:
- The M³T-CKD framework achieved a high accuracy of 94.79% in survival prediction.
- Experimental results demonstrated the effectiveness of the proposed MFD and CKD modules.
- The proposed method outperformed existing state-of-the-art approaches on a multi-modal carotid artery ultrasound dataset.
Conclusions:
- M³T-CKD offers a robust solution for survival prediction in carotid atherosclerosis by effectively integrating multi-modal data.
- The framework's ability to learn from both segmentation and survival tasks enhances predictive accuracy.
- This approach holds significant potential for improving clinical decision-making and patient management in cardiovascular disease.
Abstract:
Patients with carotid atherosclerosis are at risk for cardiovascular disease, which may lead to death. The use of ultrasound image and clinical tabular data to predict the survival of patients is of great value for the treatment and prevention of cardiovascular disease. However, there are two main challenges in survival prediction of patient carotid atherosclerosis: (1) how to effectively fuse ultrasound image and clinical tabular data, and (2) how to accurately segment plaque in ultrasound image and use it for patients survival prediction. To overcome these challenges, we propose a multi-modal multi-teacher contrastive knowledge distillation framework, called M3T-CKD, for survival prediction of patients with carotid atherosclerosis. M3T-CKD leverages teacher network trained on plaque segmentation task to assist student network learning for survival prediction of patient with carotid atherosclerosis. Specifically, we design a modality feature disentanglement (MFD) module for the teacher and student networks to learn shared and specific features of ultrasound image and clinical tabular data to realize fusion. Moreover, we propose a spatial-channel decoupling learning scheme in teacher network to address the issue of low contrast between plaques and surrounding tissues. To further utilize the plaque and background knowledge between the plaque segmentation task and survival task, we propose a novel contrastive knowledge distillation module (CKD). This module encourage the student network to learn plaque features while suppressing the learning of background features for patients survival prediction. We evaluate the performance of M3T-CKD on our collected multi-modal carotid artery ultrasound dataset. Experimental results demonstrate the efficiency of our proposed components and our network achieves the best performance with accuracy of 94.79%, and a p-value < 0.05, outperforming the state-of-the-art methods.
More Related Videos
09:11Performing Permanent Distal Middle Cerebral with Common Carotid Artery Occlusion in Aged Rats to Study Cortical Ischemia with Sustained Disability
Published on: February 23, 2016
04:09Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma
Published on: October 10, 2018
Related Concept Videos
01:30Acute Coronary Syndrome III: Diagnostic Studies
01:26Atherosclerosis III: Management