Comparative Evaluation of Advanced Deep Learning, Image-to-Text Models, and Radiomics for Predicting Tumor Budding...

This study aimed to predict tumor budding (TB) and tumor-stromal ratio (TSR), which are important parameters of the tumor microenvironment in invasive ductal carcinoma, from preoperative ultrasound images. To this end, image classification-based deep learning (DL), image-to-text-based DL, and radiomics-based machine learning (ML) approaches were compared.

We included 153 patients diagnosed with histopathologically invasive ductal carcinoma. TB and TSR were classified into two groups, "low" and "high," and separate models were developed for each dataset. Three different methodological approaches were applied: (1) advanced image classification DL models (YOLOv11x-cls, DINOv2, Vision Transformer [ViT]), (2) the Bootstrapping Language-Image Pre-training (BLIP-2) model that converts images to text, and (3) ML algorithms with radiomic features (KNN, SVM, XGBoost). All models were trained on the training set, and their performance was then evaluated on the validation and test sets.

In TB prediction, the XGBoost model demonstrated the most superior performance (AUC: 0.87, accuracy: 0.87 on the validation set; AUC: 0.76, accuracy: 0.78 on the test set). In contrast, image classification-based DL models yielded lower AUC values ranging from 0.55 to 0.71 on the validation set, while the BLIP-2 model achieved an AUC value of 0.67. In the TSR prediction, XGBoost showed the highest discriminatory ability (AUC: 0.92, accuracy: 0.92 in the validation set; AUC: 0.84, accuracy: 0.85 in the test set). In contrast, image classification-based DL models exhibited AUC values ranging from 0.54 to 0.75 in the validation set, while the BLIP-2 model exhibited an AUC of 0.65.

The findings obtained indicate that radiomics-based ML models show promise in non-invasive TB and TSR prediction using ultrasound images in breast cancer. The clinical integration of these approaches could significantly contribute to the development of personalized treatment strategies for invasive ductal carcinoma and enhance patient management.