A multi-modal deep neural network for multi-class liver cancer diagnosis

Area of Science:

• Medical Imaging and Artificial Intelligence
• Oncology and Pathology
• Deep Learning for Healthcare

Background:

• Liver disease can be asymptomatic and life-threatening.
• Accurate diagnosis of liver cancer variants and metastasis is crucial for patient outcomes.
• Current diagnostic methods may face challenges with complex datasets.

Purpose of the Study:

• To develop a multi-modal deep neural network for multi-class malignant liver diagnosis.
• To integrate computed tomography (CT) scans and pathology data for enhanced prognostication.
• To classify primary liver cancer variants and metastasis effectively.

Main Methods:

• A multi-modal deep neural network combining deep dilated convolution neural networks (for CT scans) and wide and deep networks (for pathology data).
• Utilized residual connections to mitigate vanishing gradient problems in deep learning models.
• Employed transfer learning to address insufficient and imbalanced dataset challenges.
• Concatenated hierarchical features from both data modalities for classification.

Main Results:

• The proposed model achieved an average accuracy of 96.06% in predicting three-class liver cancer variants.
• The Area Under Curve (AUC) for the classification was 0.832.
• Demonstrated superior performance compared to existing liver diagnostic studies on a benchmark dataset.

Conclusions:

• This study presents the first approach to classify liver cancer variants by integrating both pathology and image data.
• The developed multi-modal deep learning network offers a promising advancement in the medical perspective of malignant liver diagnosis.
• The model's high accuracy and AUC indicate its potential for clinical application in liver cancer diagnosis.