Semi-supervised cine cardiac MRI segmentation via joint registration and temporal attention perceiver

Segmentation of cardiac structures is essential for cardiac function evaluation using cine magnetic resonance imaging (MRI). Deep learning can be used to segment cardiac structures in cine cardiac MRI with high accuracy, but this approach requires fully annotated datasets for training, which are difficult to obtain. Semi-supervised segmentation methods provide a way to alleviate the burden of manual labeling by using labeled and unlabeled data for training. However, these methods generally provide suboptimal segmentation accuracies.

To develop a semi-supervised method that utilizes relatively small training datasets and under-annotations for improved cine cardiac MRI segmentation.

The proposed approach consists of deformable registration, fully and weakly supervised segmentation, and a temporal attention perceiver (TAP). The registration module was trained to warp labeled frames to generate pseudo labels for unlabeled frames. The warped labeled images were used to train the fully supervised segmentation network. The unlabeled images and the pseudo label were used to train the weakly supervised segmentation model, and the segmentation prediction was compared with the input pseudo label as an auxiliary loss to the registration module. The TAP module was employed to generate optimized features for the warped labeled and the original unlabeled images both paired with the original labeled image. Consistency between the resulting features was enforced to refine cross-instance feature alignment to facilitate the registration. One hundred, twenty, and ten subjects from the Automatic Cardiac Diagnosis Challenge (ACDC) and seventy-five, thirty, and fifteen cases from the Multi-Vendor & Multi-Disease (M&Ms) Cardiac Image Segmentation Challenge were used for training, each with random end-systolic (ES)/end-diastolic (ED) frames labeled. The optimized models were used to segment the remaining 50 ACDC and 50 M&Ms subjects. The proposed approach was compared with several commonly used semi-supervised segmentation methods in terms of Dice-similarity-coefficients (DSC), average-symmetric-surface-distance (ASSD), and Hausdorff-distance (HD) for left (LV) and right (RV) ventricular cavity and myocardium (Myo). A Unet trained on the same subjects each with both frames labeled was used as an upper bound (Unet_UB).

Using 100 ACDC training subjects, our approach yielded DSC = 0.910 $\pm$ 0.063, ASSD = 1.37 $\pm$ 0.63 mm, and HD = 6.38 $\pm$ 2.99 mm for RV, DSC = 0.894 $\pm$ 0.024, ASSD = 1.20 $\pm$ 1.12 mm, and HD = 4.67 $\pm$ 3.22 mm for Myo, and DSC = 0.934 $\pm$ 0.056, ASSD = 1.25 $\pm$ 1.63 mm, and HD = 3.97 $\pm$ 5.76 mm for LV. A bidirectional copy-paste (BCP) method performed the best among the comparative methods and generated DSC = 0.902 $\pm$ 0.060, ASSD = 1.45 $\pm$ 0.60 mm, and HD = 7.50 $\pm$ 3.20 mm for RV, DSC = 0.885 $\pm$ 0.030, ASSD = 1.28 $\pm$ 0.80 mm, and HD = 5.80 $\pm$ 2.80 mm for Myo, and DSC = 0.920 $\pm$ 0.068, ASSD = 1.15 $\pm$ 0.40 mm, and HD = 4.20 $\pm$ 3.30 mm for LV. For Unet_UB, these were 0.905 $\pm$ 0.068, 1.48 $\pm$ 0.61 mm, and 6.35 $\pm$ 2.85 mm for RV, 0.895 $\pm$ 0.030, 1.05 $\pm$ 0.45 mm, and 4.40 $\pm$ 3.09 mm for Myo, and 0.941 $\pm$ 0.044, 1.02 $\pm$ 0.34 mm, and 3.17 $\pm$ 1.63 mm for LV. Similar trends were observed when using 75 M&Ms training subjects. For all the experiments, our approach outperformed BCP in general and yielded segmentation accuracies comparable to Unet_UB.

The proposed approach outperformed several commonly used semi-supervised segmentation methods and yielded segmentation accuracies on par with fully supervised Unet using various relatively small datasets and under annotations for training.