Multimodal Cross-Attention Fusion of B-Mode Ultrasound and Strain Elastography for Tumor Segmentation in...

Robot-assisted minimally invasive surgery offers high precision and dexterity, opening up new possibilities for intraoperative imaging to improve tumor localization. B-mode ultrasound provides real-time anatomical views, while strain elastography (SE) adds tissue stiffness information. Used together, they provide complementary cues that help increase the accuracy of tumor localization by improving boundary detection. We present a robotics-integrated framework for autonomous ultrasound scanning and multimodal tumor segmentation. The system utilizes the da Vinci^® Classic surgical robot with the da Vinci^® Research Kit (dVRK) and a custom ultrasound transducer. It simultaneously acquires paired B-mode and SE images from silicone phantoms and ex vivo porcine livers. Our architecture extends the conventional U-Net by integrating dual encoders with cross-attention and selective fusion. Cross-attention allows one modality to refine the features of the other, while selective fusion balances their contributions. We performed experiments on a multimodal dataset to validate the effectiveness of the proposed method. Results indicate that our approach outperforms unimodal baselines and standard fusion methods across both segmentation and classification metrics. These findings suggest that multimodal cross-attention and selective fusion can improve tumor segmentation in ultrasound imaging. Ultimately, the goal is to use such advances to support intraoperative guidance for robot-assisted liver surgery.