Abdominal Ultrasound Simulation from Semantic Labels Using Paired Label-to-Physics-Based Image Translation

Current abdominal ultrasound (US) simulation methods often require CT-based anatomical references for ray-casting, limiting deformation, and pathology variability. We propose a learning-based pipeline trained to predict physics-based images derived from CT scans from semantic labels, enabling controlled simulations without patient-specific CT volumes at inference time.

We introduce a two-stage pipeline that maps anatomical segmentations to realistic US images through a simplified US image. Stage I synthesizes this image from semantic labels using models trained on CT-based ray-casting outputs. Stage II refines it into a realistic US scan using anatomically guided unpaired translation. Deformations and pathologies are generated by editing anatomical maps.

We evaluated Pix2Pix and the Semantic Diffusion Model (SDM) in Stage I, followed by segmentation-guided CycleGAN (SG-CycleGAN) refinement in Stage II. SDM significantly outperformed Pix2Pix in morphological metrics, including MAE $\downarrow$ (19.85 vs. 21.65), SSIM $\uparrow$ (0.28 vs. 0.24), and mIoU $\uparrow$ (0.43 vs. 0.29), whereas Pix2Pix yielded better perceptual point estimates (LPIPS $\downarrow$ : 0.17 vs. 0.19; FID $\downarrow$ : 0.32 vs. 0.37; KID $\downarrow$ : 0.25 vs. 0.48).

Training paired generative models with physics-based supervision enables approximation of CT-derived ray-casting outputs at inference time directly from semantic labels. Although the pipeline does not require patient-specific CT volumes at inference time, CT-derived segmentations and ray-casting simulations remain necessary to train Stage I. Once trained, the framework enables controllable healthy and pathological simulations through semantic map modification.