The pseudo-siamese framework combines Transformer and CNN for medical image generation

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• Multi-phase medical imaging aids disease diagnosis, but missing data is common.
• Synthesizing missing imaging phases is clinically significant for comprehensive analysis.

Purpose of the Study:

• To develop a novel generative adversarial network (GAN) for synthesizing missing medical image phases.
• To address limitations of existing methods that focus on either local or global features.

Main Methods:

• Proposed the siam TC-GAN, a pseudo-siamese architecture for extracting and fusing multi-scale global and local image information.
• Introduced a novel HE-loss function to improve the realism of generated grayscale features.

Main Results:

• The siam TC-GAN effectively fused local lesion details and global structural features.
• The HE-loss guided the generator to produce more realistic medical images.
• Demonstrated effectiveness on contrast-enhanced CT and contrast-enhanced MRI datasets.

Conclusions:

• The proposed siam TC-GAN with HE-loss is effective for synthesizing missing medical image phases.
• This method improves multi-modal diagnosis by addressing modality missing issues.
• The approach shows promise for enhancing medical image analysis and disease detection.