Related Experiment Video
Updated: Jul 1, 2026
03:05
Establishment and Evaluation of a Risk Prediction Model for Pathological Escalation of Gastric Low-Grade Intraepithelial Neoplasia
Published on: February 16, 2024
Interpretable deep learning-based hierarchical multi-modal fusion model for predicting HER2 expression in gastric
Chenxi Hu1,2, Changfeng Feng3, Ziyi Ye1,2
1The Fourth School of Clinical Medicine, Zhejiang Chinese Medical University, Hangzhou First People's Hospital, Hangzhou, Zhejiang, China.
Frontiers in Oncology
|June 3, 2026
Summary
A novel multimodal framework improves HER2 status prediction in gastric cancer (GC) by integrating endoscopic, radiomic, and clinical data. This approach enhances diagnostic accuracy, aiding in personalized HER2-targeted therapies.
Area of Science:
- Oncology
- Medical Imaging
- Artificial Intelligence
Background:
- Gastric cancer (GC) treatment efficacy depends on accurate HER2 (Human Epidermal growth factor Receptor 2) status determination.
- Current methods for HER2 assessment can be invasive and time-consuming.
- Developing non-invasive, accurate predictive tools is crucial for timely and personalized treatment strategies.
Purpose of the Study:
- To develop and evaluate a hierarchical multimodal framework for improved HER2 status prediction in GC patients.
- To integrate deep learning (DL) features from endoscopic images and radiomic features from CT scans with clinical data.
- To assess the performance of the multimodal model compared to unimodal approaches.
Main Methods:
- A retrospective study of 402 GC patients, with 92 having confirmed HER2 status.
- Endoscopic images analyzed with ResNet-50 for DL feature extraction and invasion depth prediction.
- Radiomic features extracted from contrast-enhanced CT using Pyradiomics, with feature selection based on reproducibility and importance.
- Six machine learning algorithms employed to build multimodal models, evaluated using AUC and sensitivity.
- SHapley Additive exPlanations (SHAP) used for feature interpretability.
Main Results:
- An invasion depth prediction model achieved a validation AUC of 0.79.
- The multimodal model integrating endoscopic, radiomic, and clinical data outperformed unimodal methods.
- Logistic Regression as the multimodal model yielded the highest validation AUC of 0.83.
- SHAP analysis identified specific radiomic and DL features as key predictors of HER2 status.
Conclusions:
- The hierarchical multimodal approach significantly enhances HER2 expression prediction in GC.
- Integration of diverse data modalities (clinical, endoscopic, radiomic) optimizes diagnostic accuracy.
- This framework supports improved clinical decision-making and facilitates tailored HER2-targeted therapies for GC patients.