Development and validation of an ultrasound-based interpretable machine learning model for the classification of
Zhicheng Du1,2,3, Fangying Fan1, Jun Ma1
1Department of Interventional Ultrasound, Fifth Medical Center of Chinese PLA General Hospital, Beijing, China.
Background:
Our study aimed to develop a machine learning (ML) model utilizing grayscale ultrasound (US) to distinguish ≤3 cm small hepatocellular carcinoma (sHCC) from non-HCC lesions.
Methods:
A total of 1052 patients with 1058 liver lesions ≤3 cm from 55 hospitals were collected between May 2017 and June 2021, and 756 liver lesions were randomly allocated into train and internal validation cohorts at a 8:2 ratio for the development and evaluation of ML models based on multilayer perceptron (MLP) and extreme gradient boosting (XGBoost) methods (ModelU utilizing US imaging features; ModelUR adding US radiomics features; ModelURC employing clinical features further). The diagnostic performance of three models was assessed in external validation cohort (312 liver lesions from 14 hospitals). The diagnostic efficacy of the optimal model was compared to that of radiologists in external validation cohort. The SHapley Additive exPlanations (SHAP) method was employed to interpret the optimal ML model by ranking feature importance. The study was registered at ClinicalTrials.gov (NCT03871140).
Findings:
ModelURC based XGBoost showed the best performance (AUC = 0.934; 95% CI: 0.894-0.974) in the internal validation cohort. In the external validation cohort, ModelURC also achieved optimal AUC (AUC = 0.899, 95% CI: 0.861-0.931). Upon conducting a subgroup analysis, no statistically significant differences were observed in the diagnostic performance of the ModelURC neither between tumor sizes of ≤2.0 cm and 2.1-3.0 cm nor across different HCC risk stratifications. ModelURC exhibited superior ability compared to all radiologists and ModelURC assistance significantly improved the diagnostic AUC for all radiologists (all P < 0.0001).
Interpretation:
A diagnostic model for sHCC was developed and validated using ML and grayscale US from large cohorts. This model significantly improved the diagnostic performance of grayscale US for sHCC compared with experts.
Funding:
This work was supported by National Key Research and Development Program of China (2022YFC2405500), Major Research Program of the National Natural Science Foundation of China (92159305), National Science Fund for Distinguished Young Scholars (82325027), Key project of National Natural Science Foundation of China (82030047), Military Fund for Geriatric Diseases (20BJZ42), National Natural Science Foundation of China Special Program (82441011). National Natural Science Foundation of China (82402280), National Natural Science Foundation of China (32171363), Key Research and Development Program for Social Development of Yunnan Science and Technology Department (202403AC100014).
