Multimodal image translation via deep learning inference model trained in video domain

Area of Science:

• Medical Imaging
• Deep Learning
• Image Translation

Background:

• Current medical image translation methods operate in the image domain.
• Medical image acquisition is a continuous temporal process.
• A novel framework is proposed to leverage the video domain for medical image translation using deep learning.

Purpose of the Study:

• To develop and demonstrate a deep learning framework for synthesizing CT images from CBCT video data.
• To evaluate the feasibility and reliability of the proposed video-domain image translation approach.
• To explore a new direction for medical image translation research.

Main Methods:

• A vid2vid framework based on conditional GANs was employed for CBCT-CT image translation in the video domain.
• Paired CBCT and CT images from 100 patients were used for supervised model training.
• A novel spatio-temporal learning objective was incorporated into the framework.
• Performance was evaluated using MAE, PSNR, NCC, and SSIM on 10 new testing patients.

Main Results:

• The framework achieved high agreement between real and synthetic CT images.
• Evaluation metrics demonstrated strong performance: MAE (23.27±5.53), PSNR (32.67±1.98), NCC (0.99±0.0059), and SSIM (0.97±0.028).
• Synthetic CT images exhibited improved quality with reduced noise and artifacts compared to CBCT.

Conclusions:

• A deep learning-based approach for medical image translation in the video domain was successfully developed.
• The framework demonstrated feasibility and reliability in CBCT-CT image translation.
• The approach is extensible to other medical imaging modalities and represents a promising new direction for the field.