Convolutional-transformer fusion of ultrasound and diffuse optical tomography for breast lesion classification

Accurate breast cancer diagnosis remains a significant challenge in clinical practice. This study presents a deep learning fusion framework that integrates high-resolution ultrasound (US) images with low-resolution diffuse optical tomography (DOT) total hemoglobin (HbT) images to improve diagnostic performance and reduce unnecessary benign biopsies. A cohort of 287 patients who underwent US and US-guided DOT imaging before biopsy of suspicious breast lesions was analyzed. A novel Convolutional-Transformer-Translator (CTT) fusion model was developed to automatically combine complementary US and DOT images for lesion classification. Standalone US and DOT models achieved mean areas under the ROC curve (AUCs) of 0.829 and 0.830, respectively, whereas the CTT model reached an AUC of 0.946, surpassing radiologist assessments using standard Breast Imaging Reporting and Data System (BI-RADS) and DOT-enhanced BI-RADS. At a matched sensitivity of 98%, the CTT model achieved biopsy specificity of 41.22%, compared with 8.59% for standard BI-RADS and 30.28% for DOT-enhanced BI-RADS evaluated by four study radiologists. The proposed CTT model was also compared with ten state-of-the-art classification models and demonstrated superior performance. These findings highlight the potential of the CTT fusion framework to improve breast lesion diagnostic accuracy and reduce benign biopsies in clinical workflows.