Related Experiment Video
Updated: Jan 9, 2026
Visualization of Vascular and Parenchymal Regeneration after 70% Partial Hepatectomy in Normal Mice
Published on: September 13, 2016
A generative vision-language model for holistic pathological assessment using preoperative imaging in hepatocellular
Liyang Wang1, Fa Tian2, Fan Li3
1School of Biomedical Engineering, Tsinghua University, Beijing, China; Hepatopancreatobiliary Center, Beijing Tsinghua Changgung Hospital, Institute for Precision Medicine, Key Laboratory of Digital Intelligence Hepatology (Ministry of Education), Tsinghua University, Beijing, China; Research Unit of Precision Hepatobiliary Surgery Paradigm, Chinese Academy of Medical Sciences, Beijing, China.
Background:
Pathological evaluation of hepatocellular carcinoma (HCC) traditionally relies on surgical resection, posing risks of infection and complications while failing to provide comprehensive pathological insights preoperatively. This study aims to develop HepaPathGPT, which utilises preoperative imaging to deliver detailed pathological interpretations, enabling non-invasive, real-time pathological assessments for patients with HCC.
Methods:
A retrospective study of 1091 patients with HCC from 10 independent cohorts was used. SegFormer-b5 segmented tumour regions, and vision-language alignment mapped imaging features to pathology descriptions. We fine-tuned four pretrained frameworks using Low-Rank Adaptation (LoRA) to efficiently translate imaging features into structured histological reports, enabling real-time evaluation via an interactive interface.
Findings:
HepaPathGPT showed robust tumour segmentation (mean Intersection over Union: 0.883 ± 0.007, Dice: 0.934 ± 0.006) and an average accuracy of 0.697 ± 0.024 for six pathological markers in external validation (n = 109). For text generation, BLEU-4 and ROUGE-1 scores were 62.7 ± 1.7 and 84.2 ± 1.1. Five pathologists rated 92.5% and 87.4% of reports as acceptable for accuracy and completeness.
Interpretation:
HepaPathGPT offers a approach for non-invasive pathological analysis in patients with HCC. This technology holds significant clinical value for decision-making in patients with HCC and promises scalability to other diseases in the future.
Funding:
National Natural Science Foundation of China (82090053, 82090052, 12326618, 82272703, 82473201); Tsinghua University Initiative Scientific Research Program of Precision Medicine (2022ZLA007); CAMS Innovation Fund for Medical Sciences (2019-I2M-5-056); Elite Youth Project of Natural Science Foundation of Fujian Province (2023J06056); Science-Health Joint Medical Scientific Research Project of Chongqing (2023MSXM092).
