Deep-Motion-Net: GNN-based volumetric liver shape reconstruction from single-view 2D projections

Area of Science:

• Medical Imaging
• Radiotherapy Physics
• Artificial Intelligence in Medicine

Background:

• Internal anatomical motion during radiotherapy complicates precise dose delivery to tumors and increases risk to surrounding healthy tissues.
• Accurate estimation and compensation for organ motion are critical for improving treatment efficacy and patient safety in external beam radiotherapy.
• Current methods often rely on surrogate signals or invasive markers, which have limitations in accuracy and patient comfort.

Purpose of the Study:

• To develop a patient-specific deep learning framework for accurate 3D organ motion prediction using only standard in-treatment planar X-ray images.
• To enable precise radiation delivery by compensating for internal anatomical motion without additional imaging modalities or invasive procedures.
• To reconstruct volumetric 3D organ models from single-view X-ray images at arbitrary angles during radiotherapy sessions.

Main Methods:

• Proposed Deep-Motion-Net, an end-to-end graph neural network (GNN) for 3D organ reconstruction from kV X-ray images.
• Utilized a 2D CNN encoder for feature extraction, followed by feature pooling and a ResNet-based graph attention network for mesh deformation.
• Trained the model using synthetically generated motion instances and kV images, incorporating digitally reconstructed radiographs (DRRs) and conditional CycleGAN for style transfer.

Main Results:

• Achieved sub-millimetre accuracy with overall mean prediction errors as low as 0.12 ± 0.11 mm on synthetic data.
• Demonstrated clinical feasibility on in-treatment kV images from four liver cancer patients, with mean peak prediction errors up to 3.29 mm.
• Validated the model's performance across various synthetic respiratory motion scenarios.

Conclusions:

• Deep-Motion-Net successfully reconstructs volumetric 3D organ models from single-view X-ray images, achieving high accuracy.
• The approach leverages accessible in-treatment imaging, offering a cost-effective alternative to MRI or invasive markers.
• This represents the first deep learning framework capable of volumetric 3D organ reconstruction from single-view images throughout an entire treatment scan series, demonstrating clinical potential.