The Study of Echocardiography of Left Ventricle Segmentation Combining Transformer and Convolutional Neural Networks

Accurate prediction of echocardiographic parameters is essential for diagnosis and treatment of cardiac disease, especially for segmentation of the left ventricle to obtain measurements such as left ventricular ejection fraction and volume. However, manually outlining left ventricle on echocardiographic images is a time-consuming and physician experience-dependent task. Therefore, it is crucial to develop an accurate and efficient automatic segmentation tool. Therefore, we aimed to explore a model to perform echocardiography of left ventricle segmentation by combining transformer and convolutional neural networks (CNN).ResNet-50 was used in CNN branch. The encoder-decoder architecture was used for transformer branch, which was fused to the corresponding feature maps of the CNN branches. Fusion module was used to effectively combine feature information from the CNN and transformer. Bridge attention used to increase sensitivity and prediction accuracy of model. The entire network was trained end-to-end using the binary cross-entropy with logits loss L.In this work, we propose an automatic left ventricular (LV) segmentation model based on Transformer and CNN that efficiently captures global dependencies and spatial details and create a fusion module using CBAM that fuses Transformer and CNN features. In addition, attention is also computed using multi-level fusion features to obtain the final attention segmentation map. The model was trained and evaluated on a large cardiac image dataset, EchoNet-Dynamic, with test dice coefficient of 92.4%.The results show that our model can better segment left ventricle. We also tested our model on clinical patient ultrasound images, and visualization results proved effectiveness of the model.