Numerical and Clinical Evaluation of the Robustness of Open-source Networks for Parallel MR Imaging Reconstruction

Deep neural networks (DNNs) for MRI reconstruction often require large datasets for training. Still, in clinical settings, the domains of datasets are diverse, and how robust DNNs are to domain differences between training and testing datasets has been an open question. Here, we numerically and clinically evaluate the generalization of the reconstruction networks across various domains under clinically practical conditions and provide practical guidance on what points to consider when selecting models for clinical application.

We compare the reconstruction performance between four network models: U-Net, the deep cascade of convolutional neural networks (DC-CNNs), Hybrid Cascade, and variational network (VarNet). We used the public multicoil dataset fastMRI for training and testing and performed a single-domain test, where the domains of the dataset used for training and testing were the same, and cross-domain tests, where the source and target domains were different. We conducted a single-domain test (Experiment 1) and cross-domain tests (Experiments 2-4), focusing on six factors (the number of images, sampling pattern, acceleration factor, noise level, contrast, and anatomical structure) both numerically and clinically.

U-Net had lower performance than the three model-based networks and was less robust to domain shifts between training and testing datasets. VarNet had the highest performance and robustness among the three model-based networks, followed by Hybrid Cascade and DC-CNN. Especially, VarNet showed high performance even with a limited number of training images (200 images/10 cases). U-Net was more robust to domain shifts concerning noise level than the other model-based networks. Hybrid Cascade showed slightly better performance and robustness than DC-CNN, except for robustness to noise-level domain shifts. The results of the clinical evaluations generally agreed with the results of the quantitative metrics.

In this study, we numerically and clinically evaluated the robustness of the publicly available networks using the multicoil data. Therefore, this study provided practical guidance for clinical applications.