Using multimodal PET+MR data as conditional generative adversarial network inputs improves pseudo-CT and attenuation...

To obtain qualitatively and quantitatively accurate positron emission tomography (PET) images, the recorded PET emission data must be corrected for photon attenuation. Attenuation correction (AC) factors are typically estimated from X-ray computed tomography (CT) data acquired during an integrated PET/CT study. Estimating these factors from magnetic resonance (MR) data in an integrated PET/MR scanner is challenging, as MR images don't provide direct information about annihilation photon attenuation. Conditional generative adversarial networks (cGANs) have shown promising results for both emission-based and MR-based AC. This study explored whether combining these approaches could further improve brain PET AC accuracy. Thirty-five patients who received same-day whole-body PET/MR and PET/CT scans participated in this study. The non-attenuation-corrected and non-scatter-corrected (NASC) PET, MR, and CT reconstructed head regions were cropped and automatically co-registered. Four networks were trained to translate NASC PET and MR images into pseudo-CTs. Three used single-modality input, and the fourth used multi-modality. The multi-modality cGAN produced significantly better pseudo-CTs vs. the single-modality cGANs, with an average structural similarity index (SSIM) and dice similarity coefficients for bone, soft-tissue, and air of 0.865±0.001, 0.715±0.002, 0.915±0.001, and 0.567±0.004, respectively, vs. 0.841±0.001, 0.660±0.003, 0.894±0.001, and 0.524±0.005, for the single-modality cGANs with the best results. When comparing the AC PET reconstructed images, all cGANs outperformed the clinical atlas-based method used in commercially available PET/MR systems, and, as expected, the multi-modal cGAN achieved the highest quality results with average SSIM, and peak signal-to-noise ratio of 0.9987±0.0001, and 50.0±0.4, respectively, vs. 0.9913±0.0024, and 44.3±0.3 for the atlas method.