A Multimodality-Contribution-Aware TripNet for Histologic Grading of Hepatocellular Carcinoma

Hepatocellular carcinoma (HCC) is a type of primary liver malignant tumor with a high recurrence rate and poor prognosis even undergoing resection or transplantation. Accurate discrimination of the histologic grades of HCC plays a critical role in the management and therapy of HCC patients. In this paper, we discuss a deep learning-based diagnostic model for HCC histologic grading with multimodal Magnetic Resonance Imaging (MRI) images to overcome the problem of limited well-annotated data and extract the discriminated fusion feature referring to the clinical diagnosis experience of radiologists. Accordingly, we propose a novel Multimodality-Contribution-Aware TripNet (MCAT) based on the metric learning and the attention-aware weighted multimodal fusion. The novelty of the method lies in the multimodality small-shot learning architecture designation and the multimodality adaptive weighted computing scheme. The comprehensive experiments are done on the clinic dataset with the well-annotation of lesion location by the professional radiologist. The experimental results show that our proposed MCAT is not only able to achieve acceptable quantitative measuring of HCC histologic grading based on the MRI sequences with small cases but also outperforms previous models in HCC histologic grading, reaching an accuracy of 84 percent, a sensitivity of 87 percent and precision of 89 percent.