HepaCopilot: A 6G-Enabled Multimodal Vision-Language Agent for Real-Time Hepatocellular Carcinoma Risk Stratification
IEEE Journal of Biomedical and Health Informatics
|May 25, 2026
Summary
HepaCopilot, an AI tool, improves hepatocellular carcinoma (HCC) detection by analyzing contrast-enhanced ultrasound (CEUS) videos and clinical data. This multimodal agent offers interpretable risk assessments, addressing variability in current diagnostic methods.
Area of Science:
- Medical Imaging
- Artificial Intelligence
- Oncology
Background:
- Hepatocellular carcinoma (HCC) is a leading cause of cancer mortality globally.
- Contrast-enhanced ultrasound (CEUS) is vital for HCC detection but suffers from interpretation variability due to operator dependence.
- Standardized interpretation of CEUS for HCC risk assessment remains a challenge in clinical practice.
Purpose of the Study:
- To develop and evaluate HepaCopilot, a multimodal vision-language agent for interpretable HCC risk assessment.
- To integrate temporal CEUS video sequences with clinical metadata for enhanced diagnostic accuracy.
- To overcome the limitations of subjective CEUS interpretation in HCC diagnosis.
Main Methods:
- Developed HepaCopilot, a multimodal AI agent utilizing Chain-of-Thought (CoT) for structured clinical reasoning.
- Employed a hierarchical temporal encoding architecture with cross-modal attention for feature extraction across arterial, portal venous, and delayed CEUS phases.
- Evaluated HepaCopilot on a public TCIA dataset of 120 HCC subjects with CEUS examinations.
Main Results:
- HepaCopilot demonstrated strong discrimination performance with an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.94 on a test set of 18 subjects.
- The AI agent systematically extracts features from multi-phase CEUS videos and integrates them with clinical metadata.
- Performance was compared against baseline methods, showing significant potential despite a wide confidence interval due to the small test set size.
Conclusions:
- HepaCopilot offers a promising AI-driven solution to enhance the accuracy and interpretability of HCC risk assessment using CEUS.
- The multimodal approach addresses the variability inherent in operator-dependent CEUS interpretation.
- Further validation on larger datasets is warranted to confirm the robustness of HepaCopilot's performance in diverse clinical settings.
