Prediction of malignancy risk in Bethesda III nodules: development and validation of multiple machine learning models

To develop and validate a machine learning (ML)-based prediction model of Bethesda III nodules and create a nomogram based on the best model.

We collected data on patients with Bethesda III nodules who were admitted between January 2020 and June 2024, including 276 Bethesda III nodules from 7371 patients who underwent ultrasound-guided fine needle aspiration (US-FNA). Clinical, ultrasonographic, cytological, laboratory, and molecular data were collected and randomly split into training and validation cohorts at a ratio of 7: 3. Six feature selection methods and ML algorithms-Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Extreme Gradient Boosting (XGB), Support Vector Machine (SVM), and K-Nearest Neighbors (KNN)-were evaluated. A nomogram was then created based on the best-performing model.

The study cohort included 276 Bethesda III nodules with a final malignancy rate of 65.2% (180/276). LR exhibited the highest area under the receiver operating characteristic (ROC) curve (AUC: 0.823) in cross-validation of the validation set. Additionally, the calibration curves and Decision Curve Analysis (DCA) results were also favorable. The model included BRAF, composition, shape, orientation, and the thyroid imaging reporting and data system (TI-RADS). The nomogram exhibited robust discrimination (AUC: 0.846 in the validation set), calibration, and clinical applicability across the two datasets after 500 bootstraps.

Among the six ML algorithms, the LR algorithm demonstrated the best performance. A nomogram was developed to predict the malignancy risk in Bethesda III nodules. This nomogram may serve as a valuable tool to reduce diagnostic uncertainty and provide personalized risk stratification for patients.