Conditional-Based Transformer Network With Learnable Queries for 4D Deformation Forecasting and Tracking.

Area of Science:

• Medical Imaging
• Radiation Oncology
• Artificial Intelligence

Background:

• Accurate dose delivery in image-guided radiation therapy (IGRT) relies on real-time motion management.
• Forecasting 4D deformations from 2D images is crucial for precise tumor targeting but faces challenges like limited dynamics and high dimensionality.
• Existing 3D tracking methods require inputs not available during real-time treatment.

Purpose of the Study:

• To develop an attention-based temporal prediction network for forecasting 4D deformations.
• To introduce a novel framework for temporal 3D local tracking using 2D cine images.
• To enhance real-time motion management in image-guided radiation therapy.

Main Methods:

• An attention-based temporal prediction network using image features as tokens.
• Learnable queries conditioned on prior knowledge for predicting future deformation representations.
• A framework for temporal 3D local tracking utilizing latent vectors to refine motion fields from 2D cine images.

Main Results:

• The proposed tracking module reduced error by 63% compared to a conditional-based transformer 4D motion model (mean error: 1.5±1.1 mm).
• The method accurately predicted future deformations in abdominal 4D MRI images (mean geometrical error: 1.2±0.7 mm).
• The approach avoids auto-regression and uses spatial transformations for image forecasting.

Conclusions:

• The developed attention-based network effectively predicts future 4D deformations from 2D cine images.
• This method offers a robust solution for real-time motion management in image-guided radiation therapy.
• The framework demonstrates significant improvements in tracking accuracy and prediction of organ motion.