Related Experiment Video
Updated: Aug 9, 2026
Real-time Video Projection in an MRI for Characterization of Neural Correlates Associated with Mirror Therapy for Phantom Limb Pain
Published on: April 20, 2019
Real-time MRI-ultrasound image translation under limited paired data using a physically motivated conditional GAN
Jiajun Chen1, Kun Chen1, Yuliang Wu2
1College of Health Science and Environmental Engineering, Shenzhen Technology University, 3002 Lantian Road, Pingshan District, Shenzhen, 518118, China.
This study introduces a fast, physically consistent framework for bidirectional magnetic resonance imaging (MRI) to ultrasound (US) image translation using a generative adversarial network (GAN). The method achieves real-time synthesis, improving image quality and computational efficiency for medical applications.
Area of Science:
- Medical Imaging
- Artificial Intelligence
- Computer Vision
Background:
- Magnetic resonance imaging (MRI) and ultrasound (US) offer complementary information but have significant appearance differences, complicating cross-modality image synthesis.
- Developing accurate and efficient MRI-US translation methods is crucial for enhancing intraoperative guidance and medical simulations.
Purpose of the Study:
- To propose a fast and physically consistent bidirectional MRI-US image translation framework using a conditional generative adversarial network (GAN).
- To enable real-time synthesis of ultrasound images from MRI data and vice versa, addressing the challenge of appearance discrepancy.
Main Methods:
- A VGG19-informed U-Net generator with residual bottleneck blocks and self-attention was employed for anatomical dependency capture.
- A multi-scale PatchGAN discriminator with spectral normalization was used to enhance texture realism and training stability.
- A composite loss function (adversarial, L1, perceptual) and a random slicing augmentation strategy for 3D consistency were utilized.
Main Results:
- The proposed framework outperformed state-of-the-art methods in perceptual realism (FID/LPIPS), structural fidelity, and ultrasound speckle statistics (ENL) on brain and prostate datasets.
- Achieved millisecond-level inference speeds (approx. 11 ms per 256x256 frame), enabling real-time image synthesis.
- Demonstrated effectiveness on both RESECT (brain) and μ-RegPro (prostate) datasets.
Conclusions:
- The framework offers an effective solution for real-time bidirectional MRI-US translation, even with limited paired data.
- It balances image quality and computational efficiency through physics-motivated augmentation and an efficient GAN architecture.
- The approach shows promise for latency-sensitive applications like ultrasound simulation and intraoperative navigation.
