Multimodal AI model for early detection of hepatocellular carcinoma

Detection of early hepatocellular carcinoma (eHCC) is important for timely treatment and improved prognosis. However, it is challenging to distinguish eHCC from pre-malignant high-grade dysplastic nodules (HGDN). Here we developed an artificial intelligence (AI) derived computational framework to identify potential biomarkers and built classification models for eHCC and HGDN. A two-stage multiscale deep learning model (TMC-net) captured the subtle features based on H&E images, and outperformed the pathology foundation model and traditional histopathological features. The learned features were consistent with clinical diagnostic criteria and could be highlighted on the virtual images, assisting junior pathologists in improving the diagnostic accuracy. Four marker genes were screened through comparative transcriptome analysis. The multimodal model based on marker genes and histopathological features achieved AUROC of 0.8875 and 0.9500 on the internal and external test sets, respectively. We confirmed the morpho-phenotype correlations of these genes and found that the multimodal features were associated with patient prognosis in a broader HCC cohort. This study reveals histopathological and transcriptomic features of eHCC, and provides an optimized AI solution for assistant diagnosis.