ROXSI: Robust Cross-Sequence Semantic Interaction for Brain Tumor Segmentation on Multi-Sequence MR Images

Deep learning-based brain tumor segmentation on multi-sequence magnetic resonance imaging (MRI) has gained widespread attention due to its great potential in supporting brain disease diagnosis. Although, compared to single-sequence images, more information is available from multi-sequence MR images, noise and artifacts on any given MR sequence can result in significant performance degradations. As in clinical routine, it is not always possible to maintain high imaging quality across all MR sequences (e.g., foreign bodies, ventricular drainage, shunts, involuntary patient motion, etc.), ensuring robustness of brain tumor segmentation from multi-sequence MR images is of great importance in clinical practice, but rarely explored. Accordingly, in this paper, we propose a robust brain tumor segmentation framework to mitigate the performance degradation caused by noise and artifacts on multi-sequence MR images. Specifically, based on semantic affinity, we propose a unique cross-sequence semantic interaction module (CSSI) to exploit inter-sequence correlations and extract noise-resilient features. In addition, we incorporate a batch-level covariance mechanism to suppress the redundant background information and improve the semantic enhancement effect of the CSSI module. In order to further improve segmentation performance, we also incorporate a sequence-level variance regularization mechanism to exploit sequence-specific features. To validate the robustness of ROXSI, brain tumor segmentation performance was evaluated under the existence of four common artifacts, at five different perturbation levels. We further performed a blinded qualitative clinical evaluation with two experienced neuro-radiologists, evaluating results from ROXSI and other popular CNN and Transformer-based segmentation models. Experimental results on two benchmark datasets demonstrate the superior robustness of ROXSI over other state-of-the-art segmentation methods.